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PREFACE 



At a time when post-secondaiy education is increasingly important to employment in 
well-paying jobs and, ultimately, to Air.erica's productivity as a nation, the failure of many 
young people to realize their full educational potential is a cause for deep concern. The 
Career Beginnings program, initiated by The Commonwealth Fund with major support from the 
Gannett Foundation and the John D. and Catherine T. MacArthur Foundation, responds to 
this concern. In sites across the country, the program identifies high school students with 
college potential who, because of their average grades and economically and/or educationally 
disadvantaged family backgrounds, might otherwise be unlikely to attend college. Then, through 
a combination of educational and employment services, it seeks to help them enroll in college 
or to find better jobs than they would ordinarily obtain. 

This study of the impacts of Career Beginnings indicates that the program succeeded 
in increasing college attendance in the year immediately following high school graduation. 
Across the seven diverse sites included in the study, the proportion of individuals targeted by 
Career Beginnings who attended either a two-year or four-year college in the post-high school 
follow-up year rose five percentage points above the corresponding rate for members of a 
control group. 

What does a difference of this magnitude mean? At this point, we lack a definitive 
answer to this question, since we do not know what proportk)n of the Career Beginnings and 
control group members will remain in college and actually graduate. We do know that college 
dropout rates are distressingly high, especially for those young people to whom Career 
Beginnings is targeted. But we also know that college graduation has a large payoff in terms 
of future earnings: For example, it has been estimated that, by age 40, black college graduates 
have earned at least $150,000 more than blacks without a college degree. Because the 
potential earnings gains are so great and the program cost relatively low. Career Beginnings 
could be a worthwhile investment even if :he five-point difference narrows over the years 
between initial college attendance and actual college graduation. 
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Career Beginnings appears to offer a coherent framework for delivering services to 
disadvantaged youths. It also seems a promising route for enhancing their post-secondaiy 
educational attainment and life opportunities. 



Judith M. Gueron 
President 
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EXECUTIVE SUMMARY 



The Career Beginnings program was initiated by The Commonwealth Fund to enhance 
the life options of urban high school students from low-income families. The Gannett 
Foundation and the John D* and Catherine T. MacArthur Foundation were major and 
signiGcant partners in the effort Launched in 1986, Career Beginnings targets high school 
juniors with average academic performance and seeks to assist them to enter college and 
upgrade their educational choices, as well as to help the non-college-bound And better jobs 
than they would otherwise obtain. 

The underlying premise is that these students usually receive little help or support from 
their families or schools in completing the steps necessary to secure admission to college. 
Ongoing technical assistance and oversight of Career Beginnings is provided by the Center for 
Human Resources at Brandeis University. This flnal report presents fmdings on the services 
provided Career Beginnings and the short-term impacts of the program. 

OVERVIEW OF THE PROGRAM AND THE STUDY 

During 1987-88 - the program's second year of operations and the period under study 
in this report - Career Beginnings operated in 24 sites, which had been chosen through a 
national competition. While the program model allows considerable flexibility and variation, 
features common to all sites include: 

• A service delivery structure invohring collaboration a'^^'^ng a local college 
or university (which serves as the program sponsor), the public schools, and 
the business conununity; 

• Jobs during the summer between the students* junior and senior years; 

• Workshops and classes during that summer and during the senior year on 
such topics as taking college entrance examinations and applying for 
financial aid, along with such activities as college and career fairs; 

• Counseling to help students make educational and career choices; and 

• Adult menton in the community, who serve as roi:^ models and actively 
assist youths in planning for the future. 
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All sites ilso targeted a similar student population. To be eligible for Career Beginnings, 
young people were expected to be enrolled in their junior year of high school, to have 
"substantially" exceeded minimum attendance requirements, to rank in the middle of their class 
academically, and to demonsUate "personal motivation and commitment beyond just school 
activities" (for example, by working part-time or participating in school or community activities). 
In addition, at a minimum, 50 percent of a site's enrollees had to be economically 
disadvanUged, 80 percent were to come from families in which neither parent had a college 
diploma, and 45 percent were to be male. 

The Manpower Demonstration Research Corporation (MDRC) was asked by The 
Commonwealth Fund and the John D. and Catherine T. MacArthur Foundation to evaluate 
the effectiveness of the program in increasing rates of college attendance and employment. 
The evaluation examined program impacts in seven sites willing to comply with the research 
requiremenU. These locations and institutions were: the Bronx, New York (Bronx Community 
GoUege): Gaiy, Indiana (Indiana University Northwest); Indianapolis, Indiana (Butler 
Umvcrsity); Jacksonville, Florida (Jacksonville University); Rochester, New York (University 
of Rochester); Santa Ana, California (Rancho Santiago Community College); and Youngstown, 
Ohio (Youngstown State University). 

The evaluation of program impacU - the difference Career Beginnings made in the lives 
of students ~ used a rigorous methodology. During the spring of 1987, each site recruited 
approximately 200 high school juniors from a number of local high schools, relying primarily 
on school personnel (principals, teachers, and guidance counselors) to identify candidates who 
were interested in the program and who met the Career Beginnings eligibility criteria. The 
sites then foUowed up with additional screening criteria and procedures, such as interviews with 
program applicants. In all, 1,574 of these young people were then randomly assigned in equal 
numbers either to the experimental group - which was encouraged to take part in Career 
Beginnings acthaties - or to the control group, whose members were exduded from Career 
Beginnings, although they were free to participate in other services available in their schools 
and communities. 

niis report is based on the experiences of the 1,233 youths who responded to two 
follow-up interviews conducted one and two years after random assignment. (For most 
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interview respondents, the first interview more or less coincided with high school graduation, 
while the second took place approximately one year after graduation.) Random assignment 
assured that there were no systematic differences at the outset between experimental and 
control group members. The experiences of the controls therefore provide a reliable 
benchmark of what would have happened to the experimental in terms of service receipt, 
college attendance, and other outcomes if Career Beginnings had not existed The differences 
between the controb' experiences and the actual outcomes for experimentak constitute the 
impacts of the services provided by Career Beginnings above and beyond the services students 
woukl have received had the program not existed, rather than the impact of these services 
compared to the alternative of no services. 

To summarize the results across the sites: Career Beginnings increased the rate of college 
attendance by experimentak in the post-high school year to 53.2 percent, an increase of 4.7 
percentage points (or 9.7 percent) over the 48.5 percent rate for controb. This average 
increase conceals a good deal of variation among the sites: The mos^ successful sites were able 
to achieve increases in college-going in excess of 10 percentage points over the control group 
rates, v^iile the least successful had minimal or even negative impacts - that is, the control 
group did as well as or even better than the experimentak. These findings suggest a number 
of implications for program improvement efforts as well as future research. 

FINDINGS ON PROGRAM IMPLEMENTATION 

• Darlag tkeir senior year of high school, controls received substantially 
More services than had been anticipated 

Career Beginnings is a coordinated package of services, many of which were individually 
available in the schools or elsewhere in the communities in which students lived. In order for 
the program to have an impact, experimental had to get more services than controls. 
Therefore, to understand the effectWeness of Career Beginnings, it is important to examine the 
extent to yAdch members of the two groups received these indhddual services. 

Table 1 shows that controls received many of the same types of specific services as their 
experimental counterparts. For example, more than half the controls reported attending classes 
on completing forms for college admission and financial aid, college fairs or other college 
information eventn, and workshops on job-readiness skills. Two-thirds met at least twice with 
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TAILE 1 



EXKRIHENTALS* AND CSNTROLS* RECEIPT OF SPECIFIC SERtlCES 
DURIN6 THE SUNNER OF TNE JUNIOR YEAR AND THE 1987-68 SCHOOL YEAR 



OutCOM 



ExpcrlMDtalt ControU 

(%) (%) Olffirtnct 



EdMatlM-teUtad Strvlets 

Htip Hith studying and 
tMt-UkIng skills 

Classts or ttorkshops to 

prtpsrt for eel logo tntranct txan 

Classts on coapltting form for 
collage adalsslon and financial aid 

Col lags fair or othtr 
col lags Informtlon avent 

Classes In reading or math, 
apart fron regular school classes 

Individual lied acadanic 

tutoring 

Received any of the six acadanic 
services listed above 

Job-lelated Service* 

Career fair or other 
Job Infonaatlon event 

Job-readlness skills workshop 
on dress, lateness, etc. 

Received either of the Job-related 
services listed above 

Other Services 



42.0 



56.0 



63.9 



60.1 



20.1 



17.6 



86.2 



S6.6 



69.1 



78.2 



24.3 



39.7 



52.8 



56.3 



13.7 



12.9 



32.5 



41.2 



56.4 



68.5 



17.7 



16. 3*** 



11. I*** 



3.8 



6.4*** 



4.7* 



3.7* 



15. 4*** 



12. 7*** 



9.7 



Mentor other than relative 
or friend 

At least two aeetlngs 
with counselor 

Class or workshop 
on faally planning 



Saaple site 



64.3 



75.0 



23.8 



621 



45.1 



66.2 



16.5 



612 



19. 2*** 



8.8 



7.3*** 



NOTES: This table Includes services received froa Career Beginnings and/or other sources. 

Calculations for this table used self-reported data for all sanplc nanbers who 
responded to both the 1988 and 1989 surveys. 

Statistical significance levels are Indicated as *** - 1 percent or less; 
*♦ • 8 percent: * ■ 10 percent. 
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t ocNmelort andi perhtpt most surprising, 45 percent reported having t mentor who was not 
t relative or friend 

Furthermore, as high school seniors, about 3S percenl of the youths in the control group 
reported having taken part in programs which, like Career Beginnings, were designed to help 
young adults with future jobs, careers, or college. 

• Noarthdess, at Mtt aites, cxpcriMitala racdved more aervicca than did 
eoatrab, altboa^ thcae dllTereacaa were aaudler thaa expected aad there 
were teportaat dlffereaces by alte la the levels of aervice receipt for both 
groapa. 

Table 1 also makes clear that rates of service receipt were higher for experimentals than 
controls for almost all the types of services considered. However, there was considerable 
variation by site in the amounts and kinds of services received by experimentak and controls, 
and in the size of the differences between the two groups. Experimentals in Gaiy and 
Jacksonville received unusually large amounts of service, convxasted not only ynih controls at 
these sites (v^ich were judged to offer relatively few programs and services to college-bound 
youths) but also with experimentak at the other sites. On the other hand, in the Bronx, an 
area judged to be relatively rich in services for youths, controk received more services than did 
oq>erimentak - mere, too, than the experimentals at several of the other sites. The 
Rochester program also did not succeed in providing more services to experimentals than they 
would otherwise have received. 

Staff of the national Career Beginnings office at Brandeis University were asked to rate 
sites according to the strength of program implementation. Their ranking corresponds closely 
with what the research showed about y/idch sites provided more services to program enroUees 
than they would otherwise have received. 

These findings on eaqperimental and control service levels mean that the results of the 
impact analysis do not compare Career Beginnings with a no-service environment Nor do they 
show the impact of services in and of themseKes. Instead, they indicate the incremental 
effects of Career Beginmngs over and above an already fairly extensive array of services - that 
is, the various services receded by the control group, which were usually, but not ahvays, fewer 
^Han those received by youths in Career Beginnings. 

• Partldpaats reported that they liked the program. 
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Nine^^ percent of thoie e3q)erimeiitab who actually participated in Career Beginninsi 
reported that the program had been veiy helpful or somewhat helpful in moving them toward 
their educatioful and emfripyment goals. Among specific components, young people found 
assis ta nc e in completing college and financial aid applications especially valuable. Ahnost 
three-quarten of those ¥dio had mentors judged this service to be very helpful or somoKiiat 
helpfiiL 

ITNDINCS ON PROCR. t IMPACTS 

• Amaa the aitca, 4MS percent of the controls attended a two- or four-year 
colk|e at aoae point daring the year after high schooL 

Nationwide, 40 percent of all 18- to 20-year-old high school graduates whose family 
inocMnes were below the poverty line and who weie not in the Armed Forces were enrolled 
in ccdlege in March 1989. Therefore, it is notable that in following the experiences of a group 
of high school juniors who were not academically gifted, many of ^^m were economically 
disadvantaged and some of whom did not graduate from high school, 49 percent were found 
to be attending college within the year following their senior year in high schooL 

Rates of college-going were unesqiectedly high even for those groups for whom a college 
education would not have been predicted on the basis of baseline characteristics. For example, 
38 percent of the controls who had ever been suspended or expelled from school went on to 
college, as did 39 percent of those with grade averages of C or less at the end of their junior 
year and 48 percent of those ^ose families were receiving cash welfare. Such findings suggest 
that Career Beginnings may have recruited a group of high school juniors who were somev^at 
more motivated than average- 

• DeapMe the rdnttvely high levd of college ennritanent anong controls, 
Cnnar llegfauin|i led to an tacrease In college attendance and, noro 
geaeroltyy to raised edncationnl aspfarations anMng experimentals. 

The proportion of ejqwrimcjtals ever attending college during the follow-up year was 
532 percent, an increase of 4.7 percentage points (or 9.7 percent) over the control group 
figure. This difference was statistically significant The impact estimate of 4.7 percentage 
points is the most conservative, since it includes all experimentals (about one-fifth of whom did 
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not piftidpate in Career Beginnings) and all controls (about one- tenth of whom received some 
services Erom the program). Adjustments for these individuals, which increase the impacts 
somewhat, are discuss^^ in the body of the report 

When vocati ^oal schook are included, approximate^ 60 percent of both cxperimentals 
and controk were counted as enrolled at some point in a post-secondaiy educational 
institution. But Career Beginnings led some who would have attended vocational schools to 
choose two- or four-year colleges instead, <.nd inay have led others who would have entered 
two-year colleges to go into bachelor's degree programs. 

• ExperiMital«€OBtrol differaiccs In college enrollment rates persisted 
thnmghovt the post*high school year. 

As Hgure 1 shows, experimentals were not only more likely than controls to go to 
college at some point during the follow-up period, but they were also more likely to begin 
college "on schedule" in the fall semester rather than later on. The experimental-control 
differential in college attendance was about 6 percentage points in September, October, and 
November, but began to narrow thereafter, as some additional controls enrolled in college 
during tlie spring term. Nonetheless, by May, the difference in the percentages of 
oq)erimentals and controls in college remained statistically significant (47.9 percent versus 43.4 
percent, respectively). 

About 90 percent of all sample members, whether experimentals or controls, who 
attended four-year colleges in the fall remained there through the spring of their freshman 
year. The retention tate in two-year colleges was 81 percent. 

• No clear ttoiy esMrges firom the data about dllfereatlal program effective- 
ness for different subgroups of the Career Beginnings popuhition. 

Women in the e3q)erimental group enrolled in college eariier than their male 
counterparts (or than women in the control group), but they were no more likely to remain 
there, once admitted By May of the follow-up year, equal percentages of men and women 
in the aq)erimental group were attending college, and xbf experimental-control differential in 
college attendance was nearly equal for both sexes. 

Few other demographic or socioeconomic characteristics (or clusters of characteristics) 
were deariy associated with program effectiveness. In contrast with some previous findings 
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FIGURE 1 

MONTHLY ATTENDANCE AT TWO- OR FOUR-YEAR 
COLLEGES. BY EXPERIMENTALS AND CONTROLS 



Percentage of Group Attending 
60 




SOURCE: Toble 4.2. 
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that interventions have been more successful with more disadvantaged groups, students with 
greater academic or economic barriers to college attendance were not helped more by the 
program. 

• There was a good deal of site variation in the program's impact on college 
nttcnduce. 

At the two sites judged to have implemented Career Beginnings most effectively, based 
on the judgment of Brandeis University staff and on the experimental-control difference in 
sendee receipt, the experimental-control differential in college attendance exceeded 10 
percentage points. In contrast, at the two sites that were judged to have been least successful, 
impacts were negative (that is, controls attended college more than experimentals). Results 
for the remaining sites fell between these poles. While these differences in impacts across sites 
are not all statistically significant, the pattern of impacts corresponds with differences in service 
receipt across the sites. 

* As expected, experimentals wor*'H less and earned less than controk 
during the foUow-up year; their forgone earnings may be seen as an 
investment in their futures. 

An immediate short-term trade-off between attending college and working was 
an*xipated and, in fact, occurred. While experimentals were more likely than youths in the 
control group to invest in their futures by enrolling in college, they were less likely ever to be 
empk>yed during the follow-up period (79.9 percent versus 84.1 percent, respectively - a 
statistically significant difference) and had lower average eamingis for the year. However, over 
the course of the follow-up year, the monthly employment and eamingis differentials 
disappeared. 

IMPUCATIONS FOR FUTURE RESEARCH AND PROGRAM DEVELOPMENT 

The findings presented above suggest several lessons, but they do not tell the final stoiy. 
One would expect^ for instance, that with time, the negative employment and eamingis impacts 
of Career Beginnings would be reversed, as experimentals leave school and move into jobs. 
It would also be important to know whether the program's early positive impacts on college 
attendance are sustained - or even increase - over time and whether college graduation rates 
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for the two groups differ. 

In the interim, this study suggests a number of key considerations for policymakers and 
for program planners and administrators. If a program like Career Beginnings is to have its 
maximum effect, sites should be chosen carefully, based on the existing level of services and 
the capacity of the program sponsor to implement the program effectively. 

First, planners need to consider the number and kinds of services that are already 
available to young people. Some communities are resource-rich - that is, they have extensive 
programs and serviceii to assist young people with their educational and career choices. In 
such areas, more attention may need to be given to coordinating and structuring services in a 
distinctive "package." Otherwise a new intervention is unlikely to make as much difference as 
would be the case if the same intervention were introduced in a relatively resource-poor area. 

Second, program implementation matters. The Career Beginnings sites that were judged 
to have implemented the program most effectively produced the largest impacts, while sites 
judged the least successful at implementation had the smallest impacts. Part of the story lies 
in staffing. The Bronx site, where program impacts were negligible, had no full-time staff at 
all, while the Jacksonville site, where impacts appeared relatively large, had both a full-time 
project director and a full-time case manager and a dozen part-time staff members as well. 

The Career Beginnings program has undergone considerable evolution. Initially, the 
program model under study in this report, as noted above, permitted a great deal of flexibility 
and variety. The national Career Beginnings office at Brandeis has since worked continuously 
with the sites to modify program activities and implementation practices. This study suggests 
the value of a more prescriptive approach, including minimum requirements to ensure that 
programs have the personnel necessary to give a new program the attention it needs. 

Lessons about ^ch youths should be recruited are more speculative. Career Beginnings 
seems to have enrolled a group of youths who were motivated both to take advantage of 
available services and to attend college. Although controls received substantial services, 
experimentals received even more, and the program was able to boost college attendance rates 
above the control group level. 

An important open policy question is whether Career Beginnings would have been 
equally or more successful if it had reached a group with more serious educational or economic 
deficits, and what changes in the program model might be needed to do this. By the spring 
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of junior year, many students v/ho might have been directed toward college have already 
dropped out of high school To affect the educational aspirations of these students, and to 
grapple more effectively with the educational and economic deGcits that often block the path 
tc college admission, it may be necessaiy to begin earlier in the lives of disadvantaged students. 
The Career Beginnings program has already embarked on this course of action. 
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CHAPTER 1 
INTRQPyCTIQN 



The Career Beginnings program, which began in 1986, is an initiative designed to assist 
"tenacious** high school juniors from low-income families to realize their educational and 
occupational potential. The program emphasizes high school completion and admission to a 
two-or four-year college; it also seeks to help non-college-bound students find jobs that develop 
their abilities. The Career Beginnings target group consists of young men and women who 
stow perseverance in their schoolwork but are not among the top students in their class. The 
underlying premise is that many of these young people come from families in which a college 
education is not regarded as realistic, and that there are gaps in their support systems, including 
supports provided by the public schools, that deter them from raising their aspirations and 
achieving their goals. 

Officials at The Commonwealth Fund initiated and originally funded the program, which 
was based on a prototype project, known as Career Explorations, developed at Hunter College 
in New York City in the early 198Qs. The initiators of the program felt that in many 
communities economically disadvantaged high school students do not receive assistance that 
would allow them to understand the options open to them and make ambitious - but realistic 
- plans to continue their education. Career Beginnings was developed to fill this perceived 
service gap. The Commonwealth Fund continues to play the leadership role and has been 
joined by the Gannett Foundation and the John D. and Catherine T. MacArthur Foundation. 

L An Ovtrview of the Program and the Study 

During 1987-88 - the program's second year operations and the period under study 
in this report - Career Beginnings operated in 24 sites, which had been chosen through a 
national competition; each served approximately 100 young people. While the program model 
aUows considerable flexibility and variation, several features are common to all sites. First is 
the service delivery structure, involving a collatoration among a local college or university 
(n^ch serves as the program sponsor), the public schools, and the business community. 
Typically, students are recruited from a number of high schook (in some cases, 10 or more), 
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allowing local program sponsors to carefully select whom to admit Second is the ser/ices 
themselves, ^ch include summer jobs, workshops and classes, and counseling. The jol» are 
provided during the summer between the students' junior and senior years. Th^ are 
supplemented with workshops and classes (during both the summer and the senior year) 
covering topics such as study skills, taking college entrance examinations, and applying for 
financial aid, as well as events such as college and career fairs. Counseling (along with related 
services) is provided during the summer and the senior year, and is designed to guide the 
students in making their educational and vocational choices. A third prominent feature of the 
program model is the use of mentors - adults in the community who not only act as role 
models but also actively assist youths in making plans for the future. 

At the national level, the program is managed by the Center for Human Resources at 
Brandeis University. The Manpower Demonstration Research C6rporatk)n (MDRC) was asked 
to evaluate the effectiveness of the program in increasing rates of school completion, college 
attendance, and employment The evaluation used a random assignment methodobgy to 
examine program impacts in seven of the 24 Career Beginnings sites across the country. The 
locations and institutions participating in the evaluation were: the Bronx, New York (Bronx 
Community College); Gary, Indiana (Indiana University Northwest); Indianapolis, Indiana 
(Butler Univecsity); Jacksonville, Florida (Jacksonville University); Rochester, New York 
(University of Rochester); Santa Ana, California (Rancho Santiago Community College); and 
Young^town, Ohio (Youngitown State University). 

The national office of Career Beginnings at Brandeis established several criteria to help 
ensure that the program would reach the young people to whom it vm directed To be 
eligible, young people were expected to: 

* Be enrolled in their junior year of high school; 

* Have "substantially" exceeded minimum school attendance requirements 
during the sophomore year, with no history of "significant disciplinary 
problems in school or in the community"; 

• Display average academic achieverr«eni, generally ranking in the middle 60 
percent of their class; and 

• Demonstrate "personal motivation and commitment beyond just school 
activities, by regularly participating in some form of school or community- 
based service, working part-time and contributing to the family, or the like." 
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Several requirements were also set at the site (rather than individual) level: 

• At least 50 percent of the site's enrollees had to be economically 
disadvantaged, as indicated by eligibility for enrollment in programs funded 
by the Job lYaining Partnership Act (JTPA), the main federally funded 
vocational training initiative; 

• At least 80 percent had to come from families in which neither parent had 
earned a college degree; and 

• At least 45 percent of the Career Beginnings participants had to be male. 

During the spring of 1987, approximately 200 eligible youths were recruited for Career 
Beginnings at each of the seven study sites, usually on the recommendation of their teachers 
and guidance counselors. The research design called for them to be randomly assigned in 
equal numbers either to an experimental group, whose members were encouraged to enroll in 
Career Beginnings, or to a control group, whose members were not permitted to enter the 
program. Members of both groups were followed up through interviews conducted one and 
two years after program entry. The first round of interviews was conducted in May and June 
of 1988, when the members of the research sample were completing their senior year of high 
school; the second round was conducted in May and June of 1989, one year after high school 
graduation for most sample members. 

An interim report based on the first round of follow-up interviews at the seven study 
sites was prepared by MDRC in 198& This is the second and final report on the evaluation, 
yMch includes information from the second round of interviews. The remainder of this first 
chapter considers the context in which Career Beginnings is unfolding, using national data to 
illuminate the program*s rationale and potential policy relevance. The second chapter discusses 
the evaluation design, presenting information on the characteristics of research sample 
members, ^pendix A, an adjunct to Chapter 2, discusses issues concerning the research 
methodology. Chapter 3 tint describes the varied implementation of Career Beginnings in the 
seven sites, and then examines the extent to which the program succeeded in delivering more 
services to enrollees than th^ would otherwise have received. Chapter 4 assesses the 
program^s impacts on college entry and retention, employment and earnings, and other 
outcomes, while Appendix B presents some additional impact findings. Appendix C looks at 
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the relationship between college entry and service receipt in general, rather than Career 
Beginnings services in particular. 

n. The PoHcv Context 

Several factors underscore the potential policy relevance of Career Beginnings. 

A number of studies have found a positive relationship between education and earnings 
(Kfincer, 1989; Smith and Welch, 1989). A recent study commissioned by The Commonwealth 
Fund focuses specifically on the benefits pnd costs of higher education for a nationally 
represenutive sample of black high school seniors (Lewin/ICF, 1990). Using data from the 
National Longitudinal Study of the High School Gass of 1972 and other sources, this work 
tracked the earnings of individuals with different amounts of post-secondary education over a 
14*year period. The authors estimated that, when measured differences in famUy background, 
work experience, and ability were controlled for, a bachelor's degree added $3,450 to $4,150 
to the 1984-85 earnings of college*educated men relative to those of high school graduates - 
an increase of 22 to 32 percent. For women, the earnings increases ranged between $3,500 
and $5,100, or 32 to 38 percent. Moreover, both black men and women with college degrees 
were more likely to be in more prestigious occupations, and to have spouses with college 
degrees and more prestigious occupations, than their counterparts who were not college 
graduates, pointing to higher household as well as individual incomes. The investigators 
concluded that getting a college education was a veiy good investment for black youths. 

The same study also examined the effects of obtaining a vocational credential rather than 
a four-year college degree. In 1984-85, men holding such a credential earned $2,800 to $3,100 
more than male high school graduates but considerably less than their college-educated peers. 
For women, having a vocational certificate did not result in earnings increases that were 
statistically signiGcant Post-secondary education that did not result in a degree or credential 
also had no significant impact on earnings for either men or women.^ 



^Caution is needed in interpreting the results for vocatbnal training because this categoiy encompasses 
training programs vaiying widely in type and duration. 

It was not possible to estimate the impact on earnings of a two-year academic degree because too 
few individuals in the National Longitudinal Survey of the High School Qass of 1972 held such a diptoma. 
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Long-term changes in the American economy are predicted to widen further the income 
gap between the well-educated and the less skilled. The Workforce 2000 report commissioned 
by the U.S. Department of Labor (Johnston and Packer* 1987) draws attention to this issue. 
It concludes that the fastest-growing jobs will be in professional, technical, and sales 
occupations requiring higher education and skills levels; jobs for the least skilled will grow 
more stowly or actually decline in number. It notes further that of the new jobs created during 
the 1964-2000 period, more than half will require some education beyond high school, and 
afanost a third will be filled by college graduates. 

Paradoxically, despite the ever-increasing importance of a college education, college-going 
by minority-group youths declined sharply through most of the 198Qs. According to a recent 
report using Census Bureau data (Carter and Wilson, 1989), the overall rate of college 
attendance rose slightly in the late 1970s and 1980s: The percentage of high school graduates 
between 18 and 24 years old from all ethnic groups and economic classes who attended a two- 
or four-year college increased from 53.4 percent in 1976 to S7.S percent in 1988. However, 
the percentage for low-income black high school graduates dropped by almost 10 percentage 
points during the same period (from 39.8 percent to 30.3 percent). For Hispanic graduates, 
the rate fell even more sharply, from 50.4 percent to 35.3 percent. As the authors of the 
report concluded, "Since the mid-1970's, the college participation of African Americans and 
Hispanics has been a picture not of progress but of major regression** (p. 5). The decline in 
college-going was especially precipitous for low-income minority males. 

There is some indication that this decline may have begun to reverse itself at the end 
of the 1980s. The Current Population Survey indicates that 40 percent of black high school 
graduates aged 18 to 20 from families with incomes below the poverty line were attending 
college in March 1969, an increase of almost 10 percentage points over the corresponding 
rate for the previous year.^ It is unclear whether this points to the beginning of a long-term 
trend. Nonetheless, the disparity between rates of college attendance for white and minority 
youths from all income levels remains pronounced and, other factors being equal, the income 
gap between whites and non-whites may widen still further. Such a development could have 



^Theie sutistics should be interpreted with care« however, owing to the small samples on whidi they 
are based. The college enrollment rate for low-income black youths was the same as for all low-income 
youths. 
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profound implications for both the economic progress and the political stability of the nation 
as a yi/bok. 

Unless a remedy is found, the problem Career Beginnings is intended to address could 
confront America well into the twenty-flrst century. In the following chapters, the report 
examines what has been learned from the Career Beginnings initiative. 

One caveat is in order The study, of necessity, presents an early and incomplete picture 
of the impacts of participation in the program. Graduating from college has much more 
important lifelong effects on earnings and other outcomes than merely being accepted to and 
starting college. An evaluation covering only the first year out of high school cannot tell the 
whok story, not only because college graduation remains some time in the future but also 
because some students may defer college for a year or longer after high school. A fuller 
accounting of the impacts of Career Beginnings would require several additional years of 
Eollow-up, to afford sample members the opportunity to graduate from college and to begin 
their post-college careers in earnest. 
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CHAPTER 2 

THE CAREER BEGINNINGS EVALUATION DESIGN 



The evaluation of Career Beginnings analyzes the effects of the program on students who 
were given access to it Since some of the services offered by the program are also available from 
other sources, the evaluation actually measures the effects of the program over and above this 
"background" level of services. The sample is made up of youths who were high school seniors 
in 1967-8& The study examines the nature of the program's services, the extent to which youths 
in the study sample received such services from Career Beginnings or other sources, the resulting 
increase in services because of the program, and the difference access to Career Beginnings made 
in immediate college attendance, earnings, and other short-term outcomes. This chapter discusses 
the study's research design and research sample, as well as methodological issues relevant to 
estimating the program's impacts. 

I. The Research Design for the Impact Study 
A. Why Random Assignment? 

To assess the impacts of Career Beginnings - the difference the program made - there 
had to be a measure of what would have happened to youths if they had not had access to it 
This applies to both the level of services they would have received without the program and their 
subsequent educational and labor market experiences. The Career Beginnings impact evaluation 
measures the effect of the observed level of service increase on subsequent rates of college-going 
and employment, and on earning^. 

The impact of Career Beginnings on college-going, for example, was the difference the 
observed increase in services made in rates of college-going. It was easy enough to measure the 
services received by Career Beginnings students and the proportion of program entrants who went 
to college - that is, the program's "outcome." However, from these "gross" rates of service 
receipt and college-going, some estimate of the rates that would have been observed without the 
program had to be subtracted to get the "net impact" of Career Beginnings. Since it was 
impossible to observe the same people both entering and not entering the Career Beginnings 
program, a comparison group of other people had to be used to measure what yvould have 



happened without the program. 

The basic problem was to identify a comparison group of individuals who were as similar 
as possible to those entering the program. Otherwise, there would be no assurance that 
differences between the groups (for example, in post-program rates of college attendance) were 
due to the program and not to other factors, such as differences in motivation and aptitude for 
academic work. Selecting a comparison group by means of random assignment is the best way 
of ensuring that there are no systematic differences between the characteristics of eq)erimental 
and comparison groups, and, therefore, that observed differences in average outcomes after 
random assignment can be attributed confidently to assignment to the program under study.^ 

Random assignment in Career Beginnings invoked identifying a larger pool of eligible 
people than the program could serve, and using a random process (the equivalent of a lottery) 
to assign them to one of two groups. The "experimental" group was permitted to enroll in Career 
B^inning^ while the "control" group could not take part in the program but was free to seek 
other services elsewhere. 

B. How Random Assignment Was Conducted 

During the period in which the research sample was drawn, random assignment was part 
of the intake process in the seven participating sites. Figure 2.1 shows that in response to 
recruitment efforts youths applied for Career Beginnings, signed an informed consent form (after 
hearing an explanation of the study), went through the process of eligibility determinatk)n, and 
were randomly assigned to either the experimental or control group. Site staff then informed the 
students of which group th^ were in. Since this notification took place some time after initial 
applicatk>n for the program and before the start-up of program acthdties, some students in the 
experimental group may have gotten invoked in other acthdties or simply k)st interest in Career 
B^ginningi; thus, although all experimentals were encouraged to participate in Career Beginningji, 
about 20 percent did not do so. 

While the timetable for recruitment, application, and random assignment varied by site, 
aU the intake into the research sample took place between February and July 1987, vdien the 
students were juniors (see Figure 12). The program started with summer jobs that year and 
continued with acthdties during 1987-88, the students* senior year. 



technical way to say that estimates of program effects have this desirable property is to call them 
"internally valid." 
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FIGURE 2.1 

INTAKE PRCX^EDURES FOR IMPACT EVALUATION SITES 
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FIGURE 2.2 

CALENDAR OF EVENTS IN THE CAREER BEGINNINGS STUDY 
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Youth* in the wmpic were followed up through two surveys, conducted over the telephone 
or, when that was not possible, in person. The first-year foIIow-up survey was conducted at the 
end of the studente' senior year to measure rates of participation in Career Beginnings and receipt 
of similar services from other sources, as well as early reports of college acceptance. TTie second- 
year follow-up survey, conducted a year later, focused on employment and attendance at coUeges 
and vocational schools. Of the 1.574 youths randomly assigned. 1.463 (93 percent) responded to 
the Gist survey. 1,277 (81 percent) responded to the second survey, and 1.233 (78 percent) 
responded to both surveys. Those who responded to both surveys - ranging from 73 percent in 
Jacksonville. Florida, to 84 percent in Gary. Indiana - formed the impact sample for the analysis 
in this report 

C Selection of Impact Study Sites 

Career Beginnings started in 1986. When the sample studied in this report was drawn 
(mid-1987), the program operated in 24 locations across the country. The impact study included 
young people from seven of these sites, which agreed to comply with the evaluation procedures: 
the Bronx. New York; Gary. Indiana; Indianapolis. Indiana; Jacksonville. Eorida; Rochester. New 
Yoric; Santa Ana. California; and Youngstown. Ohio. 

In early 1987. at the time the impact study began, national areer Beginnings program 
managers at Brandeis University considered these seven sites to be generally representative of aU 
24 kxaiUons, despite the fact that they were not chosen randomly from all the sites. However, 
dau coUected by Brandeis in the fall of 1988 (the fall after the sample's senior year) indicated 
that they were not simUar in one potentially important respect: A lower proportion of program 
terminees at the seven impact study sites reported entering college at that time than program 
tenninees at the other 17 sites (67 percent versus 74 percent). 

This does not necessarily mean that the seven sites performed worse than the others and 
that the impacts reported here are thus unrepresentative of the entire areer Beginnings program. 
Sites with tower rates of college attendance might have served youths who were less Ukely to 
attend college in the Grst place. If that were so. the programs in such sites might have made 
more of a difference - that is. their impacts might have been larger than for the programs in the 



^Asite in Bakersfield. California, that was originally slated to be a part of the evaluation '^'^PeJ 
^en ilS!^ diMovered thit a sizable number of those assigned to the control group had been allowed to 
receive program services. 
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other litei. even though their rate of college attendance was lower. However, if the lower 
•verafe rate of college attendance at the impact study sites reflected weaknesses in their 
progruns, the program impacu presented in this report are conservative estimates of the impact 
of Career Beginnings at aU sites. It is not known which of these reasons was responsible for the 
diffBient rates of coUege attendance because detailed information was not available for the 17 
sites that were not part of the impact study. 

Forms completed at sample entry provide information on the characterisUcs of the sample. 
Table 11 shows that while the sample was not uniformly disadvantaged, its characteristia did 
not suggest a particularly college-bound group. In the table, the "Sample Size" column indicates 
the number of people with the indicated characteristic, and the "ExperimenUls" and "Controls" 
columns indicate the percentage of each group with the characteristic. The "p" value column 
indicates the probability that the observed difference in characteristics between experimentals and 
controb was due to chance in drawing the sample of youths rather than systematic differences. 
(The kPHCT the "p" value, the more the difference is "statisUcally significant") For example, the 
first panel of the table - on sex of the youths - shows that sUghtly more experimentals than 
controls were female and that the probability the observed difference was due solely to chance 
was only about 12 percent^ 

TTic column labeled "Both Groups" shows the characterisUcs of the overaU study sample. 
It was about two-thirds female (somewhat higher than the target amount of 55 percent) and two- 
thirds black MuA 15 percent Hispanic. About half the sample worked in the year before random 
mignment, and nearly 90 percent were receiving mostly Bs and Cs. Only very smaU fractions of 
•ample members had ever dropped out of school, had problems with the English language, or 
were parents by their junior year of high school. About 40 percent of sample members Uved with 
both of their parents, and 40 percent had been in summer work programs at some time in the 
past SUghtly more than one-third were taking courses that were mostly coUege preparatory, and 
one-third had at least one parent who had gone to college. About one-third were receiving cash 
wcUuc Ffcwer than 20 percent had ever been expelled or suspended. 



. ^*'*« ««»P»ri«>n of experimenuls and controls is discussed in more deiaU later in this chapter and 
in Appendix A as part of the discussion of possible selection bias. 
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TABLE 2.1 

CHARACTERISTICS AT SAMPLE ENTRY. BY RESEARCH GROUP 



Both 

Sample Experi mentals Controls Groups 
Characteristic and Subgroup Size (%) (%) (%) p' 



Sex 
Female 
Male 



791 
442 



66.3 
33.7 



61.9 
38.1 



64.2 
35.8 



0.119 



Site 

The Bronx, New York 170 

Gary, Indiana 179 

Indianapolis, Indians 212 

Jacksonville, Florida 153 

Rochester, New York 174 

Santa An*, California 162 

i'c'.>f«9stown, Ohio 183 

Employed anytime during the 
year before random assignment 

Yes 646 

No 587 



13.2 
14.5 
18.0 
13.5 
13.0 
13.5 
14.2 



53.5 
46.5 



14.4 
14.5 
16.3 
11.3 
15.2 
12.7 
15.5 



51.3 
48.7 



13.8 
14.5 
17.2 
12.4 
14.1 
13.1 
14.8 



52.4 
47.6 



0.728 



0.484 



Recent grade average 
A or A- 
B* 

B or B- 
C* 

C or C- 
0* 

D or 0- 
F 



76 
183 
332 
279 
298 
33 
24 
4 



6.8 



12. 
27. 
23 
24. 

2. 



1.8 
0.0 



5.6 
17.2 
26.1 
21.6 
24.3 
2.5 
2.1 
0.7 



6.2 
14.9 
27.0 
22.7 
24.2 
2.7 
2.0 
0.3 



0.157 



Ever a school dropout 
for a semester or more 

Yes 

No 



14 

1.219 



1.1 
98.9 



1.1 
98.9 



1.1 
98.9 



1.000 



Problems with English and 
another language is usually 
spoken at home 

Yes 

No 



46 

1.187 



3.7 
96.3 



3.8 
96.2 



3.7 
96.3 



1.000 



Living with both parents 
Yes 
No 



S28 

705 



43 
56 



42. 

57. 



42.8 
57.2 



0.767 



(continued) 



TABLE 2.1 (continued) 



Both 





Sample 


Ex perl mentals 


Controls 


Groups 




Chificttrlstlc tnd Subgroup 


Si ze 


(*) 


w 


(%) 


P* 


H«s own chlld(rtn) 












Yts 


28 


1.9 


2.6 


2.3 


0.540 


No 


1,205 


98.1 


97.4 


97.7 




Evtr In « suMMr work 












progriM 












Yts 


495 


40.4 


39.9 


40.1 


0.890 


No 


738 


59.6 


60.1 


59.9 




Employtd anytime during 












tht previous school year 












Yts 


384 


31.6 


30.7 


31.1 


0.796 


No 


849 


68.4 


69.3 


68.9 




Ever expelled or suspended 












Yts 


227 


19*2 


17.6 


18.4 


0.540 


No 


1.006 


80.8 


82.4 


81.6 




Ethnicity 












Uh1tt» non-Hi sptnic 


114 


7.9 


10.6 


9.2 


0.293 


Black, non-Hispanic 


849 


70.7 


67.0 


68.9 




Hispanic 


179 


13.7 


15.4 


14.5 




Aiatrlcan Indian 


3 


0.2 


0.3 


0.2 




Asian or Pacific Islander 


86 


7.2 


6.7 


7.0 




Other 


2 


0.3 


0.0 


0.2 




Family receiving cash urelfare 












Yes 


383 


30.8 


31.4 


31.1 


0.863 


No 


850 


69. Z 


68.6 


68.9 




Age 18 or older 












Yes 


124 


9.3 


10.8 


10.1 


0.454 


No 


1.109 


90.7 


89.2 


89.9 




Courses 












General 


498 


43.4 


39.5 


41.5 


0.369 


College preparatory 


445 


35.3 


38.8 


37.1 




CoMMrclal or business 


106 


8.6 


9.1 


8.8 




Vocational or technical 


134 


10.7 


11.6 


11.2 




Other 


18 


2.0 


1.0 


1.5 




Parent (s) graduated from or 












attended college 












Yes 


405 


31.1 


34.6 


32.8 


0.204 


No 


828 


68.9 


65.4 


67.2 





(continued) 



TABLE 2.1 (contlnutd) 



Both 



Ch«r«cttr1st1c and Subgroup 


Sanpla 

Siza 


Exparlmantals 

(%) 


Control s 
(%) 


Groups 
(%) 




TmIIv r#c#1v1na Pood StanDS 
YtS 
No 


317 
916 


24.8 
75.2 


26. f 
73.4 


25.7 
74.3 


0.502 


Evtr In a yttr-round training 
program not sponsorad by tha 
sanpla mmb*r'% high school 

Yas 

No 


100 
1.133 


6.8 
93.2 


9.5 
90.5 


8.1 

91.9 


0.101 


Saaqila Siza 


1.233 


621 


612 







SOURCE: HDRC calculations from Career Beginnings enrollment form data. 



NOTES: Calculations for this table used sel f -reported data for all 621 experlmentals 
and all 612 controls who responded to both the 1988 and 1989 surveys. Subgroup sample sizes 
for some characteristics may fall short of the total number of sample members because of 
Items missing from data for some survey responders. 

Because of rounding, distributions may not total exactly 100.0 percent. 

*The colimv) labeled "p** Is the statistical significance level of the 
difference between the distributions of characteristics for the experimental and control 
groups: that Is, p Is the probability that distributions are different only because of random 
error. A Pearson chl -square test was used to test the hypothesis of equal distributions. 
Statlstica; significance levels are Indicated as « 1 percent or less; « 5 percent; 
♦ » 10 percent. 
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Sample characteristics varied widely by site, as shown in Table 12, and this must be kept 
in mind in interpreting the program impact flndings. Certain characteristics were heavily 
concentrated among people at specific sites. For example, the vast majority of sample members 
Mibo had problems with the English language were located at Sanu Ana. Thus, it is impouible 
to determine broader effects of language problems on program participation and impacts. 
Similariy, Hispanics entered the sample at Santa Ana, the Bronx, and, to a lesser extent, 
Rochester; thus, it is difGcult to separate out the effect of being Hispanic from any joint effect 
of being in one of these three locations. 

Sites showed wide variation in other characteristics as well. Of the sample members at 
Indianapolis, 87 percent had been employed during the year before sample entry, compared with 
only 33 percent of the Jacksonville sample. Seventy percent of the Rochester sample members 
had grades that were mostly G, Ds, and Fs, compared with only 25 percent of their counterparts 
in the Bronx.^ The proportion of men fell as low as 29 percent in Jacksonville and reached as 
high as 43 percent in Rochester. Santa Ana reported the highest proportion (67 percent) of 
sample members living with both parents. The vast majority (94 percent) of sample members at 
the Indianapolis site had been in summer work programs, compared with only 8 percent of those 
at Youngstown. 

Such wide variation in sample characteristics precludes attributing differences in program 
participation or outcomes entirely to differences in program organization, staff, or quality. For 
example, compared with youths at other sites, Indianapolis sample m^tmbers reported much higher 
rates of employment in the pre-program year and a low proportion of parents who had attended 
college. They may have been less inclined to go to college and thus less interested in services 
for the coUege*bound than those at other study sites. Similarly, Rochester had a high proportion 
of men and also students with poor grades - a harder-to-serve group than that in Jacksonville. 
To the extent possible, thcs^ : cress-site differences have been taken into account in the analytic 
techniques used to examine program impacts.^ 



^ couTM, grading practices may differ in different communities. 

^MttlUple regression techniques have been used to adjust for measured differences in sample members 
reOected in their respoaises to questions on the program inuke form. However, many types of possible 
differences were not and could not be - examined using this type of dau collection approach. These 
unmeasured differences, which could have included such factors as expectations about future schooling and 
emptoyment, oouM not be controlled for in this analysis. 
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TABLE 2.2 



CHARACTERISTICS AT SAMPLE ENTRY, BY SITE 











Indian- 


Jack- 


Roch- 


Santa 


Youngs- 










The Bronx » 


Gary, 


apolis, 


sonville, 


ester, 


Ana, 


town. 


All 






Sanple 


New York 


Indiana 


Indiana 


Florida 


New York 


California 


Ohio 


Sites 




CHaracterUtk and Subgroup 


Size 


(%) 


(%) 


(%) 


(%) 


(%) 


(%) 


(%) 


(%) 


p« 


Stx 
























791 


69.4 


62.0 


67.9 


71.2 


57.5 


62.3 


59.0 


64.2** 


0.047 


Nile 


442 


30.6 


38.0 


32.1 


28.8 


42.5 


37.7 


41.0 


35.8 




Enploye^ any t 1m during tht 






















jftar befort randMi asslgnnent 






















Yes 


646 


41.2 


35.8 


87.3 


33.3 


57.5 


44.4 


56.8 


52. 4*** 


0.000 


No 


587 


58.8 


64.2 


12.7 


66.7 


42.5 


55.6 


43.2 


47.6 




Recent grade average 






















A or A- 


76 


10.7 


4.5 


2.8 


5.9 


0.6 


11.2 


8.8 


6.2*** 


0.000 




183 


32.7 


10.6 


8.5 


10.5 


9.6 


17.4 


16.5 


14.9 




B or B- 


332 


31.5 


29.1 


22.2 


29.4 


19.5 


34.2 


25.3 


27.0 




C* 


279 


18.5 


24.6 


22.6 


23.5 


31.6 


12.4 


24.7 


22.7 




C or C- 


298 


3.0 


27.9 


36.8 


27.5 


29.9 


20.5 


20.9 


24.2 




IK 


33 


0.6 


2.2 


3.8 


1.3 


5.2 


2.5 


2J 


2.7 




0 or 0- 


24 


1.2 


1.1 


3.3 


2.0 


3.4 


1.2 


1.1 


2.0 




F 


4 


1.8 


0.0 


0.0 


0.0 


0.0 


0.6 


0.0 


0.3 




Ever a school dropout 






















for a seaiester or nore 






















Yes 


14 


2.4 


1.1 


2.8 


0.7 


0.0 


0.0 


0.5 


1.1* 


0.053 


No 


U219 


97.6 


98.9 


97.2 


99.3 


100.0 


100.0 


99.5 


98.9 




Problem with English and 






















another language Is usually 






















spoken at hone 






















Yes 


46 


0.6 


0.0 


0.0 


0.7 


1.7 


25.3 


0.0 


3.7*** 


0.000 


No 


M87 


99.4 


100.0 


100.0 


99.3 


98.3 


74.7 


100.0 


96.3 




Living with both parents 






















Yes 


528 


31.8 


41.3 


26.4 


37.3 


48.9 


66.7 


51.4 


42.8*** 


0.000 


No 


705 


68.2 


58.7 


73.6 


62.7 


51.1 


33.3 


48.6 


57.2 
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TABLE 2.2 (continued) 



CMracttrlttlc tnd Subgroup 



Sonplf 

sue 







IndUn- 


Jeck- 


Roch- 


Sente 


Youngs- 


All 




The Bronx, 


Cery. 


epolU» 


tonv111e» 


ester » 


Ant, 


town» 




New York 


Indf en« 


Inoiene 


T 1 on oe 


Mam VAi*k 

new lor* 


vv I 1 1 vf ri 1 9 


Ohio 


Sites 




(%) 


(%) 


(%) 


(%) 


(%) 


(%) 


(%) 


(%) 




0.6 


1.7 


4.2 


3.3 


1.7 


0.6 


3.3 


2.3 


0.128 


99.4 


98.3 


95.8 


%.7 


98.3 




on 7 
9D./ 


07 7 
Iff . / 




28.8 


50.3 


93.9 


31.4 


31.0 


24.7 


8.2 


40. I*** 


0.000 


71.2 


49.7 


6.1 


68.6 


69.0 


75.3 


91.8 


59.9 




20.6 


14.0 


37.7 


26.1 


46.6 


32.7 


38.3 


31. I*** 


0.000 


79.4 


86.0 


62.3 


73.9 


53.4 


67.3 


61.7 


68.9 




4.1 


15.6 


31.6 


19.0 


21.8 


2.5 


29.5 


18.4*** 


0.000 


95.9 


84.4 


68.4 


81.0 


78.2 


97.5 


70.5 


81.6 




3.5 


1.7 


2.8 


7.8 


16.7 


1.9 


30.1 


9.2*** 


0.000 


54.7 


96.1 


95.8 


91.5 


67.2 


5.6 


62.8 


68.9 




39.4 


1.7 


0.9 


0.0 


13.8 


45.7 


4.9 


14.5 




0.6 


0.6 


0.0 


0.0 


0.0 


0.0 


0.5 


0.2 




1.2 


0.0 


0.5 


0.7 


2.3 


46.9 


1.1 


7.0 




0.6 


0.0 


0.0 


0.0 


0.0 


0.0 


0.5 


0.2 




27.1 


36.9 


34.0 


14.4 


27.0 


34.6 


40.4 


31. 1*** 


0.000 


72.9 


63.1 


66.0 


85.6 


73.0 


65.4 


59.6 


68.9 




7.6 


10.6 


4.7 


5.9 


15.5 


21.0 


6.6 


10. 1*** 


0.000 


92.4 


89.4 


95.3 


94.1 


84.5 


79.0 


93.4 


89.9 





Nes own chnd(ren) 
Yes 
No 

Ever In e suMer work 

Yes 
No 

Employed enytlne during 
the previous school yter 

Yes 

No 

JL Ever expelled or suspended 
f Yes 
No 

Ethnicity 

Milte» non-NlspenIc 

BUck» non-HlspenIc 
. Nispenic 

Anerlctn Indltn 

AsUn or Ted f 1c Islander 

Other 

Fanlly receiving cash welfare 
Yes 
No 

Age 18 or older 
Yes 
No 



28 
U205 



495 
738 



384 
849 



227 
1»006 



114 
849 

179 
3 

86 

2 



383 
850 



124 
1,109 
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TABLE 2.2 (continued) 



Characttrittic and Subgroup 


Sample 
Sfte 


The Bronx, 
New York 
(%) 


Gary. 
Indiana 

(%) 


Indlan- 
apollt. 
Indiana 

(%) 


Jack- 
tonvllle. 
Florida 
(%) 


Roch- 
etter. 
New York 
(%) 


Santa 
Ana. 

California 
(%) 


Youngt- 
to^, 
Ohio 
(%) 


All 
Sitet 
(%) 




Conrttt 
Cantral 

Colltft prtpiratory 
Cantrcial or butinass 
Vocational or technical 
Other 


496 
445 

106 
134 
18 


46.5 
15.1 
22.6 
14.5 
1.3 


22.4 
49.4 

13.2 
13.2 
1.7 


49.0 

27.2 
9.7 

13.1 
1.0 


55.3 
34.? 
2.0 
6.0 
2.0 


49.7 
24.3 

9.5 
13.0 

3.6 


61.1 
37.7 
0.0 
0.6 
0.6 


9.9 
69.1 
4.4 

16.0 
0.6 


41. 5*** 

37.1 

8.8 
11.2 

1.5 


0.000 


Par€fit(s) graduated froM or 
attended college 

Yet 

No 


405 
828 


35.3 
64.7 


43.0 
57.0 


22.2 
77.8 


41.2 
58.8 


30.5 
69.5 


19.8 
80.2 


39.9 
60.1 


32. 8*** 

67.2 


0.000 


iMily receiving Food Stanps 
Yet 
No 


317 
916 


24.7 
75.3 


36.3 
63.7 


34.0 
66.0 


8.5 
91.5 


18.4 
81.6 


26.5 
73.5 


27.3 
72.7 


25. 7*** 
74.3 


0.000 


Ever In a year*round training 
progrM not tponsored by the 
tiMple Mrtber's high tchool 

Yet 

No 


100 
M33 


7.6 
92.4 


3.4 
96.6 


6.6 
93.4 


5.9 
94.1 


11.5 
68.5 


11.7 
88.3 


10.4 
89.6 


8.1** 
91.9 


0.030 


Sanple Sise 


1.233 


170 


179 


212 


153 


174 


162 


183 







SOURCE: NDRC calculatlont from Career Beg^nnlngt enrol liient form data. 



NOTES: Calculatlont for thft table uted telf-reported data for all 621 experlmentalt and all 612 controlt who retponded to both the 1988 and 
1989 turveyt. Subgroup temple tliet for tome characterlttlct may fall thort of the total number of sanple membert because of Items misting from data 
for tome turvey retpondert. 

Becaute of rounding, dittrlbutlont may not total exactly 100.0 percent. 

*The column labeled *p* ft the ttatlttlcal tignlflcance level of the differences among the distributions of characteristics for the 
titet: that 1t» p it the probability that dittrlbutlont are different only becaute of random error. A Pearson chi-square test was uted to tett the 
hypothetit of equal dittrlbutlont. Statittical tignlflcance levelt are Indicated as *** « 1 percent or lest; ** « 5 percent; * « 10 percent. 
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m. MethodolMlcal Issues 
A. Setectlon Bias 

Random assignment is designed to avoid "selection bias" in measuring program impacts. 
Without it, some people (for example, youths who were more or less motivated or successful 
academically) might have been more likely to have "selected" themseWes, or to have been selected 
by othen, for either the experimental or control group, thereby biasing the comparison. To 
ascertain that impacts have internal validity, it is important to determine whether random 
assignment succeeded in creating a control group with the same pre-program characteristics as 
those assigned to the experimental group and given access to Career Beginning^. 

The columns labeled "Experimentals" and "Controls" in Table 2.1 illustrate the similarity of 
the two groups on individual measured characteristics. Appendix A and Table A.1 also show that 
the two groups did not show systematic differences in measured characteristics.^ This similarity 
on measured characteristics and the underlying random process that generated the two groups 
provide support for the comparability of the two groups. 

A final issue is the similarity of experimental and controls within individual sites. As 
expected, in specific sites (where the samples were much smaller), there were such differences.^ 
Since there were some diffeiences between experimentals and controls in most sites, it was 
important to adjust for this in the analysis of program impacts at the site level. 



^In the more rigorous analysis underlying Appendix A and Table A.1 - which involves estimating a 
itgression with researdi sutus as the dependent variable and individual characteristics as independent 
variables - there was a higher chance of being assigned to the control group if one was white or had 
previously been in a year-round training program not sponsored by one^s high school. These differences 
were statistically signifiant, that is, veiy unlikely to have arisen chance when the difference does not 
exist in the population from which the sample was drawn. To summarize the similarity of the two groups, 
the analysis in Appendix A estimates that there is a 72.9 percent probability that averaU measured 
characteristia were the same for experimentals and controls among the 1,233 ^9Aio completed both waves 
of the folkiw-up survey and formed the sample for the impaa study. This high probability assures us that 
random assignment to Career Beginnings did in fact succeed in producing two comparable groups. 

^With small samples, there is less assurance that random assignment will generate exactly comparable 
groups. Just as even a "fair" coin flipped four times may not come up heads twice. With larger samples 
(or more flips of the coin) the resulu will move toward those expeaed. Appendix A presents a separate 
table of cxperimenul-contrd differences in baseline characteristics for each site (Tables A.2 through A.8). 
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B. NoareiDoiise Bias 

It if necessary to determine if the 1,233 sample members who responded to both the 1968 
and 1969 waves of the survey were representative of the entire Career Beginnings sample of 
1^4. Table A.9 in Appendix A shows that responders were significantly less likely than 
nonresponders to be male, to have entered the sample at Jacksonville, to have been emptoyed it 
any time in the previous ichoc^ year, to have had their own diildren at the time they were 
randomly assigned, or ever to have been suspended or expelled. Responders were significantly 
more likely to have been living with both parents at random assignment, and to have been in 
college preparatory programs in their high schools. 

Beca u s e of this, the results presented in this report cannot readily be generalized to the 
2L7 percent of the entire Career Beginnings sample who did not complete both survey waves. 
Since the completion rate was so high, however, the results are representative of a broad group 
of the sample. 

C Implications of Services to Controls for Impact Estimates 

As discussed in Chapter 3, some of the youths randomly assigned to the experimental 
group did not participate in Career Beginnings, and many of the controls received services similar 
to those provided to the experimental group. The basic estimates of program impacts presented 
in this report compare the experiences of all experimental (including those not served) with all 
controb (including those served).^ These estimates, therefore, are of the impact of the added 
services received by the oqperimental group above the level of services received by controls* If the 
levels of services in the two groups were similar, and the estimated impacts small, this does not 
necessarily mean that the program was ineffective. Rather, it couki mean that the impact of the 
extra ''dose'' of services received by experimental was small. 

D. Impact of Participation Versus Impact of Assignment 

Chapter 4 discusses ways in which the basic impact estimates (comparing all experimentals 
and aU controls) can be adjusted to take account of nonparticipation in the program and presents 
alternative estimates per participant. These alternative estimates depend on specific, untestable 



^As discussed later in this report, we used one-way linear analysis of oovariance to increase the 
stttistical precision of the impaa estimates and to help correct for the slight imbalances in baseline 
diaracteristics between the treatment and control group, as reported in Table 2.1 
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airanptio» about the effects of the program. As a result* these adjusted impacts are ku reliable 
than the basic impact findingi 
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CHAPTER 3 
CAREER BEGINNINGS IN OPERATION 



The Career Beginnings program model imposed a common set of requirements on the 
participating sites. All sites were expected to adopt an administrative structure that entailed 
collaboration among the sponsoring college or university, the public schools, and the business 
community. All had to recruit a group of youths meeting the program's eligibility criteria. All 
had to deliver to these young people educational and career services aimed at helping them 
enter college or get good jobs. All had to recruit and supervise adults in the community to 
serve as mentors for the youths. 

But within these broad guidelines, the model allowed for considerable diversity. The 
natbnal Career Beginnings office at Brandeis University - recognizing the differences among 
sites in their local environments, institutional arrangements, staffing patterns, and student 
populations - provided considerable advice and technical assistance but also allowed local 
program operators considerable discretion in the way they implemented the model. The 
respective roles of the three institutional partners (colleges, high schools, and businesses), the 
recruitment procedures, the kinds and frequency of services offered, the people recruited as 
mentors and the degree of oversight given to this component of the program - all these could, 
and did, differ from site to site. 

The responses of the young people were similarly varied. Teens participated in the 
program as a whole, and in specific program activities, at different rates at the seven sites. 
Different implementation practices and rates of participation, together with differences in the 
chai^ .teristics of program participants and the local environment (including the availability of 
services for college-bound youths outside of Career Beginnings), resulted in differences among 
the sites in the extent to which Career Beginnings participants received more services than 
would otherwise have been the case. 

This chapter explores several aspects of program implementation. The early sections 
present data that are largely qualitative in nature. The discussion begins with a brief overview 
of the local program environments, followed by an examination of the process by which young 
people were recruited for the program. Next, the chapter examines the extent to which those 
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randomly assigned to the experimental group actually participated in Career Beginnings and 
those assigned to the controJ group were successfully excluded from the program. Then 
attention turns to the services themselves and how they were delivered, followed by 
participanu* assessments of the helpfulness of the program. 

The later sections of the chapter focus on the kinds and amounts of services received 
by members of the experimental and control groups in Career Beginnings and elsewhere in 
their community, as measured by the interviews conducted one and two years after the youths 
had entered the research sample. The issue addressed is a critical one because the ability of 
any impact analysis to detect differences in outcomes between experimentals and controls rests 
on the assumption that members of the two groups vnW receive different treatments. 
Otherwise, the impact of the special services under study will be diluted, if not absent 
altogether. 

The basic story that emerges from these findings is that at most sites, although Career 
Beginners did receive more services than youths who were not in the program, the difference 
was less than expected because youths in the control group received many more services than 
program planners had anticipated. The fact that controls received substantial assistance is 
considered in the discussion of program impacts in Chapter 4. 

I The Program Cnntnrt 

As Table 3.1 shows, the Career Beginnings programs under study were located in settings 
that varied widely in terms of population, ethnic composition, unemployment rates, percentage 
of the population in poverty, and other characteristics. 

Every community is, of coursr;, different. For example, those who live in one community 
may have a very different idea of the kind and amount of education needed for success in life 
than those who live elsewhere. Li communities where minority youth unemployment rates are 
high (as in the Bronx, Gary, Rochester, and Youngstown), a college degree may be seen as 
essential to getting a well-paying job, and participation in programs to help youths secure 
admission to college might be expected to be high. On the other hand, in communities in 
which it is easier to find high-paying jobs without special credentials, fewer students will seek 
a post-secondary education or the services that would help them get such an education. 

Similarly, some communities offer a range of educational and career resources to youths, 
while others (.-;otably, Santa Ana among the Career Beginnings sites) have relatively few 
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TABLE 3.1 

LOCAL CONTEXT OF CAREER BEGINNINGS PROGRAMS 



Oiaracttiistlc 



The Bronx. 
New York 



G«ry, Indlanepolls, Jacksonville, Rochester, Senta Ana, YoungstoMn, 
Indiana Indiana Florida New York California Ohio 



I 

I 



Total 

Population: 1986 

Black and 
Hispanic 

Population: 1980 (%) 

Nlnorl ty 
Educational 
AttalniMnt*: 1980 (%) 

Poverty 
Rate: 1979 (%) 

Minority Youth 
UntMploywent 
Rate": 1980 (%) 

Other College and 
Career Services 
Available to 

High School Youth 



1.194.000 



63.7 



46.2 



27.6 



32.5 



Many 



137.000 



77.5 



55.8 



20.4 



39.8 



720.000 



22.5 



52.8 



11.5 



27.4 



610.000 



26.9 



50.1 



16.0 



27.8 



236.000 



30.7 



45.7 



17.5 



36.2 



Few 



Many 



Some 



Some 



237.000 



48.3 



30.5 



14.0 



9.6 



Few 



105.000 



36.4 



49.7 



18.2 



42.7 



Some 



S''''IRCES: U.S. Bureau of the Census, 1987; U.S. Bureau of the Census. 1961; Brandeis University Career Beginnings National 
PrograM Office site reports and personal ccmnunl cations between the National Program Office and HORC. 

NOTES: All calculations are based on central city data except the Bronx calculations, which are based on county data. 
'Percentage of black end Hispanic adults age 25 or older who have at least a high school diploma. 
''Percentai^ie of unemployed black and Hispanic youths age 16-19. 
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programs for economically disadvantaged, college-bound youths. One might expect that the 
introduction of a program like Career Beginnings would make more of a difference where 
existing programs are scarce than where they are relatively more available. 

In Chapter 2, variations in the characteristics of sample members at the Career 
Beginnings sites were noted. Variations in the local contexts in which the Career Beginnings 
programs are kxlged must also be borne in mind in interpreting site-specific data on program 
participation and impacts. Indeed, it is possible to view the program impact finding! presented 
in this report as reflecting the interplay of three factors: the characteristics of the population 
eligible and selected for Career Beginnings at the site, the social and economic characteristics 
and existing service levels of the surrounding community, and the strength of the Career 
Beginningis program itself. 

n* The Progmin Strarture 

In each local program, an institution of higher Iciiming served as the program sponsor; 
among the seven in this study, four are publicly funded and three are private (see Table 3^). 
Five offer bachelor's degrees; two are two-year colleges. Th^ vary also in degree of selectivity, 
from community colleges that accept all eligible applicants to one (the University of Rochester) 
that accepts only a fraction of those who apply. Sponsorship by colleges was one reason most 
local programs emphasized the goal of post-secondary education over securing better 
employment for youths after they completed high school. Tb^ numbc; of high schools from 
vfidch students were recruited and selected ranged from 4 in Jacksonville to 13 in the Bronx. 

Staffing patterns also varied from site to site. The Jacksonville program appeau to have 
been especially richly staffed, with a full-time project director, full-time case manager, and i2 
part-time staff during the school year. At the other end of the spectrum, the Bronx site had 
no full-t«jne staff at all (and, in fact, was the only program without either a full-time project 
director or a full-time program coordinator or manager). The Santa Ana site was notable for 
its fuU-time mentor coordinator. 

Although strong support from the sponsoring educational institution had been an 
iraportant criterion in the initial competition to select Career Beginnings sites, this support did 
not always remain solid. The Youngstown program was discontinued after the study year 
because Youngstown State University no longer wished to provide funding. 
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TABLE 3.2 

CHARACTERISTICS OF CAREER BEGINNINGS PROGRAM SPONSORS 



Cli«r«cttr1st1c 



Tht Bronx, 
Ntw York 



6«ry, Indian! 



Indianapolis, 
Indiana 



Jacksonville, 
Florida 



I 
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PrograM 

Sponsor 

Public or 
Privatt 

Institution 

2- or 4-Ytar 

Institution 



of 

High Schools 
PartlcliMtlng In 
Carttr Btglnnlngs 

Carotr 

Btglnnlngs 
Staff 

ConplSMnt^ 



Naturt of 

Institutional 
Support for 
Project 
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Bronx CoMMinlty 
Col logo 



Public 



Indiana Unlvtrslty 
Northwest 



Public 



2-Year 



4-Year 



12 

Part-time Project 

01 rector 
Part-time Project 

Coordinator 
4 Part-time Counselors 



6 Full -time Employees 

Including: 
Part-time Project 

01 rector 
Full-time Project 

Manager 



Strong 



Strong 



Butler University 



Private 



4-Year 



Full-time Project 

01 rector 
4 Part-time Project 

Coordinators 
4 Part-time Interns 



Moderate 



Jacksonville University 



Private 



4-Year 



Full-time Project 

01 rector 
Full-time Case Manager 
12 Part-time Employees 

(Counselor, Tutors, 

Instructors) 
9 Sunnier Staff 



Strong 
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TABLE 3.2 (continued) 





Rochtsttr, 


Santa Ana* 


Youngstown, 


Chtracttrlstlc 


New York 


California 


Ohio 




Unlvarsltv of 


Rancho Santlaao 


Younjstoitfn State 


Sponsor 


Rochester 


Community College 


University 


Piihl 1 c or 

r UV 1 1 W Vf 








Pri Vdttt 








Institution 


Private 


Public 


Public 


2- or 4-Yt«r 








Inalll luvion 


^"1 cor 


2-Yaar 

fc * Wl 


4-Year 


Miliar Af 








High Schools 








r«rxi Ci pcx 1 09 in 








ucrwr ovyinnin^a 






5 


Carttr 


Part-tliae Project 


Part-time Project 


Part-time Project 


BmiI imliMi^ 

0V«i 1 lift 1 fly 9 


D1 ractor 


01 rector 


01 rector 


Staff 


Fu11*t1iie Project 


Full-time Project 


Full -time Project 


COBOl MMnt' 


Coordinator 


Coordinator 


Coord 1 nator 


1 Other Full -time 


4 Part-time Counselors 


2 Full-time Case 




and 3 Other Part- 


Full -time Nentor/ 


Managers 




time Employees 


Industry Specialist 


2 Part-time Volunteers 


Nature of 








Institutional 








Support for 








Projtct 


Moderate 


Strong 


Weak 



SOURCES: Brandels University Career Beginnings National Program Office site reports and personal conmunl cations between the 
National Program Office and HORC. 



NOTE: ^Excludes clerical staff. 



in. PirtkliMmt Rccniltment 

While the national Career Beginnings office established the basic parameters of program 
eligibility - including junior-year status, a satisfactory attendance record, average grades, 
personal motivation, economic disadvantage, and parents who had not attended college - the 
sites were given complete discretion in determining just which individuals who met these 
criteria should enter the program (or, more accurately, the random assignment pool, from which 
a random pick of program entrants was made). Since sites were recruiting their sample 
members from up to 12 high schools (in that case, an average of 17 students per school), they 
could be selective in setting enrollment requirements. 

In recruiting participants, it appears that the sites were attentive to the stipulation that 
enrollees show "personal motivation and commitment.** Most relied on school personnel 
(principals, teachers, and guidance counselors) to nominate candidates who met the basic 
eligibility standards and then followed up with additional screening criteria and procedures. For 
instance. Gar}' expected students to demonstrate a strong interest in post-secondary education 
or vocational training. Five of the seven sites interviewed young people applying to the 
program, and several of these imposed additional requirements. Bronx Community College, for 
instance, would not interview teens unless they had submitted written applications that included 
&Tk autobiographical essay;^ Jacksonville and Youngstown involved the teens* parents, and 
Jacksonville requi**'*'^ recommendations from teachers, ministen, or others. 

Once the impact study began, the random assignment research design obligated sites to 
recruit twice as many teens (half of whom would be placed in the control group) as they had 
the first year of program operations. Although some sites found this burdensome, their basic 
response was to intensify rather than alter the recruitment measures they had previously 
adopted.^ 



^It spears that the Bronx site largely recruited youths eligible for Career Beginnings from two other 
programs for ooUege-bound high school students operated by the community college. This may have 
weakened Career Beginnings* identity as a separate entity for young people at that site, as well as inaeased 
the likelihood of high levels of service receipt among controls. 

^Indianapolis was one exception. In recruiting the project's first-year participants, Butler Unwersity 
sttff had required applicants to come for an interview at the university itself, rather than at their high 
schools, and had regarded showing up for the interview as a sign of commitment to the program. The 
volume of applicants in the second year made these on-site interviews infeasible. Suff believed that as 
a result, the second-year group induded in this study was, as a whole, less motivated than their first-year 
counterparts. 
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This screening process may have produced a group of teens who were somewhat more 
motivated than average students to participate in activities aimed at helping them move foiward 
in their lives. The programs wanted to ensure that young people entering Career Beginnings 
would take full advantage of the sendees offered. But the random selection of the control 
group ensured that it would also be composed of young people who wanted to make something 
of themselves and would be persistent in seeking services. This characteristic of both 
experimentals and controls must be borne in mind in reading the remainder of this report. 

IV. Overall Rates of Participation in Career Beginnings 

Once students had been recruited and randomly assigned, it was important to preserve 
the integrity of the research groups by ensuring that those in the control group did not receive 
the Career Beginnings services that were supposed to be limited to experimentak. Across the 
sites, about 11 percent of the controls reported in the first-year interviews that th^ had taken 
part in services or activities associated with the program.^ This relatively high percent reflects 
the difficulty of maintaining the intended difference between experimentals and controls when 
services are provided within a school attended by both groups, as was the case in some sites. 
It is important to note, however, that 71 percent of the controk reporting some participation 
in Career Beginnings also noted that they had taken part in two or fewer activities.^ This rate 
and intensity of control group members* self-reported participatbn in Career Beginnings 
corresponds with information received from site staff. 

Table 33 shows that, at eveiy site, there was a large difference uciween the proportions 
of experimentals and controls who reported receding Career Beginnings services. The 
differences in Career Beginnings participation ranged from 48 percent in the Bronx program 
to 81 percent in Gaiy, Indiana. The column in the table labeled "p" indicates the probability 
that the observed differences in participation between experimentals and controk was due to 



^AU data on participation in programs are from the follow-up surveys rather than program 
administrative records. 

^Higher than average rates of control participation were reported in Santa Ana and the Bronx. In 
Santa Ana, where the enrollees included a sizable oontir^nt of students of Southeast Asian origin, there 
is some indication that the controk who reported receiving Career Beginnings services may not have 
undeistood these particular questions, bi ihe Bronx, half the controk who reported reoehdng any Career 
Beginniap services said they had received more than two types of services. In thk regard, it k worth 
recalling that Bronx Community College operated a number of high school-to-college transition programs, 
and the dktinctions among these may not have been clear to all the students. 

-30- 



ERLC 



63 



TABLE 3.3 

PARTICIPATION IN THE CAREER BEGINNINGS PROGRAM, 
BY SITE AND RESEARCH GROUP 





Sample 


Experimental s 


Controls 








Site 


Size 


{%) 


(%) 


Di f f erence 


p« 




Th* Bv>Anv Nftw York 


170 


70.7 


22.7 


48.0*** 


0.000 




Gary, Indiana 


1/9 


91 . 1 


in 1 


Ol .u 






Indianapolis* Indiana 


212 


72.3 


6.0 


66.3*** 


0.000 




Jacksonville* Florida 


153 


72.6 


8.7 


63.9*** 


0.000 




Rochester* New York 


174 


80.2 


8.6 


71.6*** 


0.000 




Santa Ana* California 


162 


85.7 


16.7 


69.0*** 


0.000 




Youngstown* Ohio 


183 


85.2 


7.4 


77.8*** 


0.000 




All Sites 


1.233 


79.5 


11.3 


68.2*** 


0.000 





SOURCES: MDRC calculations from Career Beginnings enrollment form and 1988 survey 

data. 



NOTES: Calculations for this table used self-reported data for all 621 experlmentals 
and all 612 controls who responded to both the 1988 and 1989 surveys. Participation in 
Career Beginnings was measured by affirmative answers to 1988 survey questions. 

Because of rounding* there may be slight discrepancies In reported sums and 
differences of the experimental and control proportions. 

*The column labeled "p" Is the statistical significance level of the 
difference between experimental and control proportions: that is, p is the probability that 
observed proportions are different only because of random error. In this table* all p 
values are below 0.0005 and thu** reported as 0.000 when rounded to three decimal places. A 
two-tailed t-test was used to test the hypothesis of equal proportions. Statistical 
significance levels are Indicated as *** « 1 percent or less; ** « 5 percent; 
* « 10 percent. 
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chtDoe in drawing the sample of youths for the study; the lower the p value, the more the 
difference is statistically significant. Here, since the differences are very large, the p values 
round to zero. 

Table 33 also shows that not all those assigned to the experimental group took part 
in the program. Across the sites, about 80 percent of the experimental said that they had 
participated in Career Begiiinings; this figure ranged from 71 percent in the Bronx to 91 
percent in Gary. In part, this less-than-complete participation may have reflected the fact that 
random assignment took place some time before tbe start-up of program activities (as noted 
in Chapter 2). In the interim, the teens may have opted to participate in other activities or 
simply lost interest 

V. Site Variation s in Service OfTerings 

What it meant to be in Career Beginnings * what activities took place, where, and with 
what frequency - varied considerably from s/.c to site. Table 3.4 shows this variation, 
highlighting the key features of five illustrative program components. 

F6r example. Career Beginnings began with a summer job complemented by academic 
remediation and/or enrichment, emplpyability and career exploration, and recreational activities. 
The Jacksonville summer program was unique in that Career Beginners lived for six weeks in 
Jacksonville University dorms, thereby getting a real taste of campus life. At the other sites, 
participants worked either four days a week, with a fifth day devoted to other activities, or (to 
accommodate the desires of local employers), five days a week, with ancillary activities taking 
place in the evenings and on weekends. The Indianapolis site, to build spirit and identification 
with the program, divided participants into teams and scheduled biweekly team meetings in 
addition to the larger group events. 

Acthdties during the school year also varied in the frequency with which they were 
conducted and in whether they typically were offered at students* high schools or on the 
campus of the sponsoring institution. For instance, as the third colunm in Table 3.4 shows, 
weekly workshops providing information on how to prepare for college were offered in Gary 
and Indianapolis at the high schools, while in Jacksonville, they took place at the university. 
Santa Ana and Rochester also held these workshops on-campus, but less frequently (Santa 
Ana monthly, and Rochester three times over the course of the year). Workshops to prepare 
students for college entrance exams (the fourth column) were generally held on-campus, but 
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TABLE 3.4 



SPECIAL FEATURES IN THE DELIVERY OF SELECTED CAREER BEGINNINGS SERVICES, 

BY SITE: 19B7-1988 



Site 


SuHMr Fnri chnent 

wWRmVI bill 1 VilHVfVIIV 

Activities 


Col 1 eclft PrftDarat 1 on 

W 1 1 V f 1 1 W 1 VII 

Information Sessions 


Preparation for 

Col 1 Fntranr* 

Exams 


CMr00T FvnlorJiflon 

VQl CCf 1 vf 1 VII 

Activities 


Case Management 


The Bronx, 

Umt York 


Orl entatlon, 

V III ^ll*^Utl^ 

iiorkshops, college 
visits 


On-campus, using 

• ivii ui cuniission* 

and financial aid 
offices 


On-campus or 

•UUSI U1 £ VU pri VQiC 

course 


Monthly meetings 
vfi in counsel urs 


Not Implemented 


Gary, 


Orientation by JTPA 
svaiTt 0 weeKaay 
workshops 


Weekly, at high 
school s 


On-campus 


A few workshops. 
TieiG irips 


Not Implemented 


• 

IndlintpolISi 
Indian* 


Evening and Saturday 
workshops, biweekly 
"team meetings" 


Weekly, at high 
schools 


On-campus, but also 
at high schools 


Career fair; also 
included In meetings 
with mentors and 
counselors 


Monthly counselor 
contact (In person 
or by phone) 


Jacksonville^ 
Florida 


6-week on*»campus 
living experience 
combining work, 

counsel 1 ng 


Weekly, on-campus 


At high schools 
(open to non-Career 
Beginnings students) 


On-campus workshops 


Weekly small group 
meetings 


Rochester, 
New York 


Week-long orienta- 
tion before work 

VApCI IVllWC, 

3 workshops 


3 on-campus 
workshops 


Not offered by 
program; referrals 

to hiah srhAol or 

lll>|ll 9^llwU 1 VI 

other Classen 


Workshops In high 
schools 


Not Implemented 


Santa Ana, 
California 


6 "fifth-day- 
workshops combining 
academic work, 
career planning 


Nonthly, on-campus 


On-campus 


On-campus workshops 


Monthly Individual 
and small group 
meetings, weekly 
monitoring 


Youngstoim, 
Ohio 


3-day orientation. 
3 1-day workshops 


At high schools 


On-campus 


Workshops in high 
schools 


Monthly counselor 
contact (In person 
or by phone) 



SOURCES: Brandels University Career Beginnings National Program Office site reports. 



Bronx participants could also enroll in a private course (the cost of which was defrayed by the 
program); the Rochester program did rot offer this service directly, instead referring Career 
Beginners to workshops offered by their high schools. 

The staff of the national office increasingly realized the importance of case management 
and counseling for individual participants. As the sixth column shows, they judged three of 
the sites (the Bronx, Gary, and Rochester) to be unable to implement an effective case 
management component during the study year. The other sites differed in both the frequency 
(weekly or monthly) and type (small group or individual, telephone or in-person contact) of 
case management efforts. 

If other components wer? selected for scrutiny, the same message of variation would 
emerge. (Sites differed, for example, in the amount of training they provided to mentors and 
the degree of supervision of mentor-student relationships.) As the national Career Beginnings 
oGGce concluded in analyzing the study sites* senior-year activities, the sites yielded "the 
impression of a . . . mosaic of differing approaches to implement these components" (Career 
Beginnings National Program Office, 1988). 

What the data in Table 3.4 do not convey is a sense of the overall "quality" of each 
program, as gauged by the amount of attention given to the program by supervisory personnel, 
the degree of innovation, and the ability of staff to relate to and motivate participants. 
National Career Beginnings office staff, who regularly monitored program activities, developed 
their own informal rating^. In their view, the Gary and Jacksonville uites were especially strong. 
At the other end of the spectrum, they judged the Bronx and Rochester sites to be quite weak. 

VI. Participants* Responses to the Program 

Members of the cscperimental group who actually participated in Career Beginnings were 
asked the extent to whicli they considered the program to have helped them move toward their 
educational and employm<int goals. Table 3.S shows their responses. Almost half (46 percent) 
said that the program had beer very helpful to them in planning their future after high school, 
and another 44 percent described it as somewhat helpful. Overall, only 10 percent thought that 
the program had not helped them at all. Teens in Gary and Santa Ana responded especially 
favorably to Career Beginnings; teens in the Bronx and Indianapoli were somewhat more 
negative than tneir counterparts elsewhere, but even at these sites, the large majority regarded 
the program as at least somewhat helpful to them. 
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TABLE 3.5 



PARTICIPANTS' VIEWS ON PROGRAN'S HELPFULNESS 
IN MOVING TOWARD THEIR EDUCATIONAL AND EMPLOYMENT GOALS: 
PERCENTAGE DISTRIBUTION OF EXPERIMENTAL GROUP PARTICIPANTS. BY SITE 



Vtry Somtwhat Not At All All 
Sample Htlpful Htlpful Helpful Categories 



Site 


Size 


(*) 


(%) 


(*) 


(*) 




The Bronx, New York 


58 


36.2 


43.1 


20.7 


100. 0*** 


o.oco 


Gary. Indiana 


81 


58.0 


37.0 


4.9 


100. 0*** 


0.000 


Indianapo1i;<. Indiana 


81 


43.2 


42.0 


14.8 


100. O**" 


0.000 


Jacksonville,, Florida 


61 


47.5 


44.3 


8.2 


100. 0*** 


0.000 


Rochester. New York 


65 


46.2 


44.6 


9.2 


100. O*" 


0.000 


Santa Ana. California 


72 


51.4 


43.1 


5.6 


100. 0*** 


0.000 


Youngstown. Ohio 


74 


36.5 


S6.8 


6.8 


100. o*** 


0.000 


All Sites 


492 


45.9 


44.3 


9.8 


100. 0*** 


0.000 



SOURCES: MORC calculations from Career Beginnings enrollment form and 1989 survey data. 

NOTES: Calculations for this table used data for all 494 experimental s who responded to 
both the 1988 and 1989 surveys and who reported participation In the Career Beginnings program. 
Sample sizes reported may fall short of this number because of Items missing from data for some 
survey responders. 

Because of rounding, rows may not total exactly 100.0 percent. 

'The column labeled "p" is the statistical significance level of the differences 
among category proportions: that Is, p is the probability that observed proportions are 
different only because of random error. In this table, all p values are below O.OOOS and thus 
reported as 0.000 when rounded to three decimal places. A Pearson chi -square test was used to 
test the hypothesis of equal proportions. Statistical significance levels are indicated as 
*** ■ 1 percent or less; ■ 5 percent; * ■ 10 percent. 



Career Beginnings participants were also asked to assess the helpfulness of the speciGc 
program components that comprise the Career Beginnings model. In general, as seen in Table 
3.6, the large majority of those who received any given service found it helpful (although, as 
noted below, participants were much less likely to receive some services than others). Most 
of the participants (81 percent) received assistance in completing college and Gnancial aid 
applications, and almost half that number (39 percent of the total) believed it was very helpful. 
Mentoring was the second most frequently received service: More than 75 percent of the 
participants had mentors, and almost three*quarten of those who did (56 percent of the total) 
found this service either very helpful or somewhat helpful. 

It is clear, then, that most participants looked positively upon their experiences in the 
program. 

Vn. Service Receipt bv Experimcntals and Controls 

Did Career Beginnings lead to higher levels of service receipt than would have occurred 
without the program? The remainder of the chapter examines this issue, using several 
approaches to compare the experiences of experimental and control gn^up members. The 
discussion starts by considering the amount of participation in special programs other than 
Career Beginnings while the study group was still in high school. Next, data on the receipt of 
individual services are presented. Then, differences in service receipt among sample members 
at the seven sites under study are probed. Finally, service utilization in the post-high school 
follow-up year is discussed. 

A. Partldpatlon in Other Special Programs 

As part of the follow-up interview, young people were asked whether they had 
participated in programs other than Career Beginnings that were designed to help young adults 
with future jobs, careers, or college. (The question further noted that these programs or 
acthdties might be offered by high schools, community groups, churches, or other organizations.) 
The top panel of Table 3.7 presents the results. It shows that ac:oss the sites, similar 
proportions of experimentals and controls took part in such programs (37 percent of the 
experimentals versus 35 percent of the controls). Only in Santa Ana was there a statistically 
significant difference between the two groups. Rates of control group participation in special 
programs were especially high in the Bronx, Indianapolis, and Jacksonville. 
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TABLE 3.6 

PARTICIPANTS' VIEWS ON HELPFULNESS OF SPECIFIC SERVICES: 
PERCENTAGE DISTRIBUTION OF EXPERIMENTAL GROUP PARTICIPANTS 



I 

Of 

I 



ERIC 



ProgrM Strvlct 



Vtry SoMMhat Not At All Old Not All 
SMpIt Helpful Htlpful Holpful Participate Catagorles 



Sizt 



30.8 

31.1 

39.4 

23.2 

15.9 
15.7 



Mtcatlon-llilatetf Strvlcts 
H«1p with studying and 

tttt-taking tklllt 490 

Clatttt or iwrkthops to 
prtpart for colltgt antranct txams 492 

Classtt on coMplttlng forms for 
colltgt adnlttlon and financial aid 492 

Colltgt fair or other 

colltgt InforiMtlon tvtnt 492 

Classts In rtading or nath, 
apart fron rtgular school classes 492 

Indlvlduallztd acadenic tutoring 492 

Job-Rtlattd Strvlcts 

Carttr fair or othtr 

Job InforiMtlon tvtnt 491 26.3 

Job-rtadlntss skills workshop 

on drtss, lateness, etc. 491 32.0 

Othtr Strvlcts 

Ntntor othtr than rtlatlvt 

or fritnd 492 31.9 

At Itast two meetings 

with counselor 492 32.7 

Class or trarkshop on 

family planning 492 13.4 



28.8 

28.5 

27.0 

23.6 

17.7 
11.6 

29.1 
25.9 

24.2 
25.8 
18.7 



13.3 

13.0 

14.4 

14.6 

13.2 
14.2 

11.8 
12.0 

20.5 
12.6 
14.8 



27.1 

27.4 

19.1 

38.6 

53.3 
58.5 

32.8 
30.1 

23.4 
28.9 
53.0 



100.0* 



100. 0*** 

100. O*" 

100.0*** 
100.0*** 

100.0*** 
100.0*** 

100.0*** 



100.0* 



0.000 



100.0*** 0.000 



0.000 

0.000 

0.000 
0.000 

0.000 
0.000 

0.003 



100.0*** 0.000 



0.000 
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TABLE 3.6 (continued) 



SOURCES: NMC calcuUtlons from Carttr Boglnnlngs tnrollinent fonn and 1989 survey dele. 

NOTES: This table Includes services received from Career Beginnings end/or other sources. 

CelcuUtlons for this table used data f^r all 494 experimental s i«ho responded to both the 1988 and 1989 surveys 
and Mho reported participation In the Career Beginnings program. Sample sizes reported may fall short of this number because 
of Items missing from data for some survey responders. 

Bacause of rounding, rows may not total exactly 100.0 percent. 

•The column labeled "p" Is the statistical significance level of the differences among category proportions: 
that Is, p Is the probability that observed proportions are different only because of random error. A Pearson chl-square 
test was used to test the hypothesis of equal proportions. Statistical significance levels are Indicated as > 1 percent 
or less; » 5 percent; ♦ » 10 percent. 



74 



TABLE 3.7 



PARTICIPATION IN PROGRAMS OFFERING EDUCATION- AND JOB-RELATED SERVICES, 
BY SITE AND RESEARCH GROUP: 1987-1988 



outcoM tnci dixt 


Sampi e 

dize 


Experimental s Controls 
(*) (*) 


T\\ f f Arftnf A 


n* 

r 


Actlvt In programs other than Career 
























The Bronx 1 New York 


170 


47.6 


40.9 


J& 7 




Gary, Indiana 


179 


35.6 


32.6 


3.0 


0.07/ 


IndlanapolISi Indiana 


212 


42.9 


39.0 


3.9 


0.9/1 


Jacksonville, Florida 


153 


29.8 


42.0 




0. 110 


Rochester, New York 


174 


30.9 


35.5 


-4.6 


A 

O.dZZ 


Santa Ana, California 


162 






11.8* 


A AO 7 

0.087 


Youngs town, Ohio 


183 


40.9 


31.6 


9.3 


A 1 A 1 

0.191 


All sites 


1,Z33 


37.2 


34.5 


9 7 
<•/ 




Active In Career Beginnings and/or 












other programs 












The Bronx, New York 


170 


81.7 


52.3 


29.4*** 


0.000 


Gary, Indiana 


179 


93.3 


34.8 


58. 5*** 


0.000 


Indianapolis, Indiana 


212 


80.4 


42.0 


38.4*** 


0.000 


Jacksonville, Florida 


153 


79.8 


44.9 


34.9*** 


0.000 


Rochester, New York 


174 


81.5 


39.8 


41. 7*** 


0.000 


Santa Ana, California 


162 


88.1 


29.5 


58. 6*** 


0.000 


Youngstown, Ohio 


183 


88.6 


35.8 


52. 8*** 


0.000 


All sites 


1.233 


84.7 


39.9 ' 


44.8*** 


0.000 



SOURCES: M)RC calculations from Career Beginnings enrollment form and 1988 survey data. 



NOTES: Calculations for this table used self-reported data for all 621 experlmentals and 
all 612 controls who responded to both the 1988 and 1989 surveys. Participation In Career 
Beginnings and/or other programs was measured by affinnatlve answers to 1988 survey questions. 

Because of rounding, there may be slight discrepancies In reported sums and 
differences of the experimental and control proportions. 

'The column labeled '^p'* Is the statistical significance level of the difference 
between experimental and control proportions: that is, p Is the probability that observed 
proportions are different only because of random error. A two*ta11ed t«test was used to test the 
hypothesis of equal proportions. Statistical significance levels are Indicated as 
■ 1 percent or less; *♦ « 5 percent; ♦ « 10 percent. 
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The bottom panel of the table indicates that on a site-by-site basis, from 30 to 52 
percent of the controls, and from 82 to 93 percent of the experimentals, were in either Career 
Beginnings or another program. At every site, the experimental-control difference was large 
and highly statistically significant. Since, as noted above, experimental and controls participated 
at similar rates in other programs, it is safe to conclude that Career Beginnings itself made the 
difference - that is, that Career Beginnings significantly increased the extent of participation 
in special educational and employment-oriented programs for teens in the experimental group.^ 
At the same time, the data also show that the increment in service provision for the Career 
Beginnings target population was not as large as program planners originally expected, since 
more than one-third of the controls did make use of alternative programs. 

B» Receipt of Specific Services 

Table 3.8 considers the extent to which experimentals and controls received the specific 
services that make up Career Beginnings, whether in Career Beginnings itself, in other 
programs, at their schools, or elsewhere. 

One caveat is need in presenting these data: Career Beginnings should not be viewed 
merely as an aggregation of separate services. In the view of personnel in both the national 
Career Beginnings office and the local sites, the program's purpose was not simply to offer 
services, but to weave through these services a set of messages and themes aimed at enhancing 
participants' feelings of self-worth and their ability to plan for the future. Thus, the table is 
not in itself an adequate gauge of the program's implementation success. Nonetheless, one 
would expect impacts to be undercut if Career Beginners and control group members received 
very similar levels of individual services. 

The table shows that controls did receive services, but the level of service receipt for 
experimentals was higher for most activities, and these differences were statistically significant. 
The mentoring component, an important part of Career Beginnings, showed the largest 
difference: 64 percent of the experimentals reported having a mentor, while 45 percent of the 



^It is worth noting that the percentage of teens in the experimental group who reported participation 
in Career Beginnings and/or another special program was not much larger than the percentage of teens 
in Career Beginnings (see Table 3.3). While 80 percent of the teens in the experimenul group reported 
participating in Career Beginnings and 37 percent reported being in another program, 85 percent were in 
Career Beginnings and/or another program. Thus, one can conclude that most teens in the experimenul 
group who took part in other programs did so in addition to their Career Beginnings activities. 
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TABLE 3.8 



RECEIPT OF SPECIFIC SERVICES, 
BY RESEARCH GROUP: 1987-1986 



OMtCOM 



Ei«c«tlM«lltUtad Strvlcts 

Htip with studying and 
ttst-taking skills 

Classts or workshops to 

prtp«rt for colltgt tntrance exams 

Classes on conplatlng forms for 
collage admission and financial aid 

College fair or other 
college Information event 

Classes In reading or math, 
apart from regular school classes 

Individualized academic 
tutorl ng 

Received any of the six academic 
services listed above 

Job*Re1ated Services 

Career fair or other 
job Information event 

Job-readiness skills workshop 
on dress, lateness, etc. 



Other Services 

Nentor other than relative 
or friend 

At least two meetings 
with counselor 

Class or workshop 
on family planning 



Smiple Size 



Experimental s Control s 



(%) (%) Difference p* 

42.0 24.3 17. 7*** 0.000 

56.0 39.7 16. 3*** 0.000 
63.9 52.8 11. I*** 0.000 

60.1 56.3 3.8 0.181 

20.1 13.7 6.4*** 0.003 
17.6 12.9 4.7** 0.023 

86.2 82.5 3.7* 0.077 

56.6 41.2 15. 4*** 0.000 

69.1 56.4 12. 7*** 0.000 

68.5 9.7*** 0.000 

64.3 45.1 19. 2*** 0.000 

75.0 66.2 8.8*** 0.001 

23.8 16.5 7.3*** 0.001 

621 612 



Received either of the job-related 
services listed above 78.2 



EMC 



(continued) 



TABLE 3.8 (contlnutd) 



SOURCES: NORC calculations from Carttr Beginnings tnrollMnt form and 1988 survty data. 

NOTES: This tabit Includes strvlcts rtctlvtd from Carttr Beginnings and/or othtr sourcts. 

Calculatl'^ns for this tabit ustd stif-rtporttd data for all 621 txptrlmentals and 
all 612 controls Mho n i ponded to both the 1988 and 1989 surveys. Receipt of specific services 
was neasurtd by afflnMtlve answers to 1988 survey questions. 

Because of rounding, there may be slight discrepancies In reported sums and 
differences of the experimental and control proportions. 

'The column labeled **p** Is the statistical significance level of the difference 
between experimental and control proportions: that is, p Is the probability that observed 
proportions are different only because of random error. A two-tailed t-test was used to test the 
hypothesis of equal proportions. Statistical significance levels are indicated as 
« 1 percent or less; ■ 5 percent; * ■ 10 percent. 
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controls also noted that they had someone other than a parent, another relative, or a close 
personal friend who took a special interest in helping them plan for future jobs or schooling. 
Other large differences in service receipt included: (1) help with studying and test*taldng skills 
(42 percent to 24 percent), (2) workshops on how to take the SATs and other college entrance 
examinations (56 percent to 40 percent), and (3) a career fair or other job information event 
(57 percent to 41 percent). Relatively few members of either group were likely to participate 
in remedial or enrichment classes apart from regular school, to receive tutoring, or to attend 
a family planning workshop, but experimental-control differences persisted in these areas as 
weU. 

The table also groups services into three categories: education-related (the largest 
category), job-related, and other career services. More than 80 percent of both experimental 
and controls took part in at least one activity aimed at helping them to succeed academically 
or to enter college, and more than two-thirds of each group attended an employment-related 
activity. 

Overall, then, the message is that while more experimentals participated in the activities 
that make up the Career Beginnings model than did their control group counterparts, young 
people in both groups received substantial amounts of service. 

C Site Vartottons in Service Receipt 

There was considerable variation by site in the amounts and kinds of services received 
by experimentals and controls, and in the size of the differences in services between the two 
groups. 

For each of the seven sites under study. Table 3.9 shows the percentages of 
experimentals and controls who received, from any source, zero to 11 of the different services 
listed in Table 3.8, as well as the average number of services received by members of each 
group. Two findings stand out First, e3q)erimentals in Gary and Jacksonville received 
unusually large amounts of service, contrasted not only with controls at these sites but also with 
experimentals at the other sites. Second, differences in the mean number of services received 
by experimentals and controls at the Bronx and Rochester sites were not statistically significant, 
indicating that the Career Beginning$ programs at these sites did not provide more services to 
experimentals than th^ would otherwise have received (Bronx controls received more services 
than those at any other site and, indeed, more than experimentals at several sites.) 



TABLE 3.9 



PERCENTAGE DISTRIBUTION OF SAMPLE MEMBERS. 
BY SITE. RESEARCH GROUP. AND NUMBER OF SERVICES RECEIVED: 1987-1988 



Number of Servleas 





Sinpit 


None 


1-2 


3-5 


6-8 


9-11 


Total 


Mtan Numbtr 


Sitt and RtstLrch Group 


Sizt 




(*) 


(%) 


(*) 


(*) 


(%) 


of Strvicts 


Tht Bronx, Nw York 


















Exptrlntntals 


B2 


2.4 


14.6 


37.8 


29.3 


15.9 


100.0* 


5.2 


Controls 


88 


0.0 


11.4 


31.8 


47.7 


9.1 


100.0 


5.6 


Gary, Indiana 


















Exptrlntntals 


90 


1.1 


6.7 


20.0 


53.3 


18.9 


100. o*** 


6.5*** 


Controls 


89 


1.1 


22.5 


36.0 


34.8 


5.6 


100.0 


4.8 


Indianapolis, Indiana 


















Exptrlntntals 


112 


6.3 


20.5 


33.0 


26.8 


13.4 


100. o*** 


4.9*** 


Controls 


100 


5.0 


26.0 


51.0 


17.0 


1.0 


100.0 


3.7 


Jacksonvlllt, Florida 


















Exptrlntntals 


84 


1.2 


9.5 


32.1 


31.0 


26.2 


100. o*** 


6.2*** 


Controls 


69 


2.9 


26.1 


43.5 


15.9 


11.6 


100.0 


4.4 


Rochtsttr, NtM York 


















Experliiental s 


81 


1.2 


23.5 


39.5 


29.6 


6.2 


100.0 


4.6 


Controls 


93 


4.3 


23.7 


39.8 


30.1 


2.2 


100.0 


4.1 


Santa Ana, California 


















Exptrlntntals 


84 


3.6 


9.5 


36.9 


40.5 


9.5 


100. o*** 


5.5*** 


Controls 


78 


7.7 


32.1 


35.9 


21.8 


2.6 


100.0 


3.7 


Youngs town, Ohio 


















Exptrlntntals 


88 


1.1 


15.9 


28.4 


40.9 


13.6 


100. 0*** 


5.5*** 


Controls 


9S 


9.5 


25.3 


46.3 


17.9 


1.1 


100.0 


3.5 


All sitts 


















Exptrlntntals 


621 


2.6 


14.5 


32.4 


35.7 


14.8 


100. o*** 


5.5*** 


Controls 


612 


4.4 


23.7 


40.8 


26.6 


4.4 


100.0 


4.2 



SOURCES: NORC calculations from Carttr Btglnnlngs tnrol^-^^nt form and 1988 survty data. 

NOTES: This tabit Includts strvicts rtctlvtd from Carttr Btglnnlngs and/or othtr sourcts. 

Calculations for this tabit ustd stif-rtporttd data for all 621 txptrlmtntals and all 612 
controls Mho rtspondtd to both tht 1988 and 1989 survtys. Participation In sptclfic strvicts was 
ntasurtd by afflmatlvt answtrs to 1988 survty qutstlons. 

Btcaust of rounding, rows may not total txactly 100.0 ptrctnt. 

A Ptarson chl-squart ttst was ustd to ttst tht hypothtsis of tqual distributions for 
txptrlmtntals and controls. Statistical signlflcanct Itvtis art Indicattd as > 1 ptrctnt or Itss; 
** ■ 5 ptrctnt; ♦ ■ 10 ptrctnt. 
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Table 3.10 examines the extent to which members of the two research groups at the 
different sites received, from any source, six key services for which the experimental-control 
differential across sites was especially large (see Table 3.8): mentoring, study and test-taking 
skills, preparing for college entrance examinations, applying to college and to get financial aid, 
gaining exposure to different careers, and learning about empbyers' expectations for conduct 
on the job. 

The table shows, first, that the proportion of teens in the experimental gioup who 
reported receiving a given service varied considerably from site to site. For example, the 
percentage reporting having mentors ranged from 42 percent in the Bronx to 75 percent in 
Rochester. Similarly, the proportion attending a career fair or other event to inform attendees 
about various occupational choices ranged from 43 percent in the Bronx to 67 percent in 
Gaiy. 

To some extent, these differences can be related to differences in the implementation 
of the components across the sites. As indicated in Table 3.3, the Rochester Career 
Beginnings program did not itself offer classes preparing participants for college entrance 
exams, instead referring them to activities operated by the public schools; and the proportion 
of Rochester experimental who indicated that they had attended a class or workshop in this 
area was lower than at any other site. By contrast, in the Bronx, participants could attend 
either workshops given by the college or a private course; there, a relative^ high percentage 
of enrollees received this service. 

In other cases, the relationship between implementation and participation is not 
straightforward. For example, one might expect that the absence of an effective case 
management component would result in lower levek of service receipt. This would seem to 
be confirmed by th experience of the Bronx and Rochester sites, which were judged not to 
have put case management in place, and which were marked by relatively low rates of 
participation in many activities. On the other hand, the Gary site, which also did not 
implement case management, led the other sites in the percentage of experimentals receiving 
many of the services examined. 

Table 3.10 also indicates that the percentages of controls receiving a given service varied, 
although on most of the indicators, the range was smaller for controk than for experimentals. 
Control group teens in the Bronx and Gary received especially large amounts of service, while 
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TABLE 3.10 



RECEIPT OF SELECTED SERVICES, 
BY SITE AND RESEARCH GROUP: 19S7-1988 



Samplt Experl mentals Controls 



OUICOM crKI diwt 


Sua 


(*) 


(*) 


01 f ftrence 


p" 


Had Mmtor othtr than rtlatlve 












or frltnd 












Tha Bronx, Naw York 


170 


41.5 


52.3 


-10.8 


0.160 


Gary, Indiana 


179 


66.7 


56.2 


10.5 


0.151 


Indianapolis, Indiana 


212 


S8.0 


45.0 


13.0* 


0.058 


Jacksonvllla, Florida 


153 


71.4 


53.6 


17. 8" 


0.023 


Rochastar, Naiv York 


174 


75.3 


48.4 


26. 9*" 


0.000 


Santa Ana, California 


162 


66.7 


17.9 


48.8*" 


0.000 


Youngs town, Ohio 


183 


71.6 


41.1 


30. 5*** 


0.000 


All A i A 

All sitas 


1,233 


64.3 


45.1 


19.2*** 


0.000 


Recalvad halp with studying and 












tast-taking skills 












Tha Bronx, Naw York 


170 


42.7 


43.2 


-0.5 


0.948 


Gary, Indiana 


178 


64.4 


35.2 


29. 2*" 


0.000 


Indianapolis, Indiana 


211 


33.9 


14.1 


19. 8"* 


0.001 


Jacksonvliia, Florida 


153 


52.4 


26.1 


26. 3*** 


0.001 


Kochastar, Naw York 


174 


19.8 


19.4 


0.4 


0.948 


Santa Ana, California 


162 


28.6 


15.4 


13.2** 


0.044 


YoungstOMn, Ohio 


183 


52.3 


17.9 


34. 4*** 


0.000 


All A i A 

All Sitas 


1,231 


42.0 


24.3 


17. 7"* 


0.000 


Atttndad classas or workshops to 












prtpart for collaga antranca axams 












Tha Bronx, Maw York 


170 


64 6 


55.7 


8.9 


0.236 


Gary, Indiana 


179 


76.7 


48.3 


28. 4*** 


0.000 


Indianapolis, Indiana 


211 


49.5 


36.0 


13. 5** 


0.048 


Jacksonvliia, Florida 


153 


53.6 


43.5 


10.1 


0.217 


Rochastar, Naw York 


174 


35.8 


37.6 


1.8 


0.804 


Santa Ana, California 


161 


54.8 


28.6 


26. 


0.001 


Youngs town, Ohio 


182 


56.8 


28.7 


28. 1"* 


0.000 


All idtas 


1,230 


56.0 


39.7 


16. 3*** 


0.000 



(continued) 
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TABLE 3.10 (contlnutd) 



Suipl t Exptrlmtntal s Control s 
OutcoM and Sitt Sizt (%) (%) Differtnct 



Atttndid classts on coiaplttlng form 
for collogt amission and financial aid 



Tht Brmx, NtM York 


170 


48.8 


60.2 


-11.4 


0.136 


Gary, Indiana 


179 


67.6 


50.6 


17. 2** 


0.019 


Indianapolis, Indiana 


212 


67.0 


59.0 


6.0 


0.232 


Jacksonvllla, Florida 


1S3 


64.3 


47.6 


16. 5** 


0.041 


Rochtttar, NtM York 


174 


51.9 


52.7 


-0.6 


0.913 


Santa Ana, California 


IfiS 


73.6 


55.1 


16.7** 


0.013 


Youngstown, Ohio 


163 


71.6 


43.2 


26.4*** 


0.000 


All sitas 


1,233 


63.9 


52.6 


11.1*** 


0.000 



Atttndtd a carter fair or other 
job InfonMtlon event 



Tha Bronx, Naw York 


170 


42.7 


52.3 


-9.6 


0.213 


Gary, Ind1<-,ia 


179 


66.7 


49.4 


17.3** 


0.019 


Indianapolis, Indiana 


212 


47.3 


26.0 


19.3*** 


0.004 


Jacksonvllla, Florida 


1S3 


63.1 


46.4 


16.7** 


0.039 


Rochastar, Naw York 


174 


54.3 


31.2 


23.1*** 


0.002 


Santa Ana, California 


162 


59.5 


52.6 


6.9 


0.376 


YoungstoMn, Ohio 


16?. 


64.4 


33.7 


30.7*** 


0.000 


All sitas 


1.232 


56.6 


41.2 


15.4*** 


0.000 



Attended a Job-read 1 nets skills 
Morkshop on dress, lateness, etc. 



Tha Bronx, Naw York 


170 


46.3 


60.2 


-13.9* 


0.070 


Gary, Indiana 


179 


92.2 


64.0 


26.2*** 


0.000 


Indianapolis, Indiana 


212 


63.4 


57.0 


6.4 


0.345 


Jacksonvllla, Florida 


1S3 


70.2 


47.8 


22.4*** 


0.005 


Rochastar, Naw York 


173 


65.0 


58.1 


6.9 


0.353 


Santa Ana, Callfoml* 


162 


71.4 


47.4 


24.0*** 


0.002 


Youngstown, Ohio 


182 


74.7 


56.8 


17.9** 


0.011 


All sitas 


1.231 


69.1 


56.4 


12.7*** 


0.000 



SOURCES: NDRC :a1cu1at1ons from Career Beginnings enrollment form and ^988 survey data. 



NOTES: This table Includes services received from Career Beginnings and/or other sources. 
Calculations for this table used self -reported data for all 621 experimental s and 
all 612 controls who respoi >d to both the 1988 and 1989 surveys. Participation In selected 
services was measured by aft^nMtlve answers to 1988 survey questions. Sample sizes may fall 
short of the total number of sample members because of Items missing from data for some survey 
responders* 

Because of rounding, there My be slight discrepancies In reported sums and 
differences of the experimental end control proportions. 

*The column labeled 'p' Is the statistical significance level of the difference 
between experimental and control proportions: that Is, p Is the probability that observed 
proportions are different only because of random error. A two-tailed t-test was used to test the 
hypothesis of equal proportions. Statistical significance levels are Indicated as 
*^ ■ 1 percent or less; • 5 percent; * » 10 percent. 
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their counterparts in Indianapolis, Santa Ana, and Youngstown received relatively Uttle 
assistance. 

Finally, as would be anticipated from the preceding discussion, the cxperimentalK»ntrol 
differential in service receipt varied a great deal among the sites. For example, in Santa Ana, 
three and a half times as many experimentals as controls (67 percent versus 18 percent) 
reported having mentors. In Gary, Rochester, and Youngstown, the proportion of 
experimentals with mentors was the same as or slightly higher than in Santa Ana, but because 
more controls also had mentors, the experimental-control differential at these three sites was 
not as large as in Santa Ana. The Bronx site ranked second-highest (surpassed only by Gary) 
in the proportion of experimentals receiving preparation for taking college entrance exams, but 
the percentage of Bronx controls receiving this assistance was also higher than at any other site. 
As a result, there was no statistically significant difference between the proportions of Bronx 
experimentals and controls receiving this service. 

Significantly higher proportions of experimentals than controb at the Youngstown site 
received all six of the services listed in Table 3.10. At Gary, Jacbonville, and Santa Ana, 
higher proportions of experimentals received five of the six services. Rochester experimentals 
were more likely than controls to have had mentors and to have attended a career fair, but 
were not more likely to have gotten the other four kinds of services. Bronx expcrimentak 
fared the won,t; in fact, they were less likely than controls to receive all the types of services 
indicated. 

TTiese data help to provide a statistical handle on program "quality." In assessing 
• "quality," ii is critical to take into account not only the level of service receipt (that is, the 
percentage of enrollees taking part in a given activity or set of activities) but also the strenffh 
of the treatment (that is, whether program participation results in substantially more service 
than would otherv^e be the case). It is particularly noteworthy that the national Career 
Beginnings office's ranking of sites (discussed in an earlier section) corresponds to a 
considerable degree with the ranking of sita according to their ability to provide more services 
to program enrollees than they would otherwise have received. That ranking rated the 
programs in Gary and Jacksonville as especially strong and the Bronx and Rochester to have 
serious weaknesses. Nonetheless, it is important to remember that many factors other than the 
way in which Career Beginnings was implemented - including the characteristics of the Urget 
popuh ion and of the surrounding community - can have a critical effect on service receipt. 
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D. VarlattoMS to Service Receipt bv Subgroup 

Besides program site, were there other factors that affected the amount of service 
sample members received? In particular, were the demographic, socioeconomic, or school- 
related characteristics of sample members at baseline associated with subsequent service 
utilization? 

Table 3.11 compares the mean number of different services received by experimentals 
and controls with different baseline characteristics.^ These baseline characteristics are organized 
into categories, which have different subgroups. To take an obvious example, the two 
subgroups under the category "Sex** are female and male; other subgroups reported in the 
table are those who were employed in the year prior to entry into Career Beginnings and those 
who were not, those who had ever dropped out of school for a semester or more and those 
who had not, and so forth. 

The table addresses two key issues: Within which subgroups were experimental-control 
differences in service receipt statistically significant? And, were some subgroups more likely 
than others to receive mc/e services? 

The experimental-control differential in service receipt is shown in the column of the 
table labeled ''Subgroup Impact," while the probability that differential could have arisen by 
chance appears in the column to the right labeled 'p". Within most subgroups, experimentals 
received a significantly larger average number oi serve/*' than did controls. For example, 
women in the experimental group received an average of 5.5 services, while women in the 
control group received an average of 42 services; men in the experimental and control groups 
received, on average, 5.5 and 4.3 services, respectively. Experimentals who had never dropped 
out of high school received, on average, 5.5 services, compared to the average of 4.2 services 
received by controk who had never dropped out There are only a few exceptions to the 



service receipt dau in this Uble, unlike the other ubles in the chapter, are statistically adjusted. 
This is necessary because ^n the sample is partitioned into subgroups, the other characteristics of sample 
memben may not be randomly distributed across these subgroups. For example, if men are more likely 
to drop out of high school than women, unadjusted dau on service receipt for men and women would 
reflect not only the differential by sex but also the dififerential by prior dropout sutus. Sutistical 
adjustments treat the da^^ or I/the dropout rate (as well as any other differences) were equally distributed 
between men and women, thereby making it possible to isolate the specific effect of sex on service receipt. 
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TABLE 3.11 



IMPACTS ON AVERAGE NUMBER OF SERVICES RECEIVED, 
BY SELECTED BASELINE CHARACTERISTICS 



Characttrlstic and Subgroup 



Sample 
Size 



Average Number of 
Services Received 

During 1987-19B8 

Experimental s Controls 



Subgroup 
Impact 



Subgroup 
Impact 
Difference'' p* 



Sex 

Fanale 791 
Hale 442 

Site 

The Bronx, New York 170 

Gary, Indiana 179 

Indianapolis, Indiana 212 

Jacksonville, Florida 153 

Rochester, New York 174 

Santa Ana, California 162 

Youngstown, Ohio 183 

Employed anytime during the 
year before random assignment 
Yes 646 

No 587 

Recent grade average 

A or B 595 
C, 0, or F 638 

Ever a school dropout 
for a semester or more 
Yes 14 

No 1.219 

Problons with Engll'h and 
another language Is usually 
spoken at home 
Yes 46 

No 1,187 

Living with both parents 
Yts 528 
No 705 



5.5 
5.5 



5.2 
6.3 
4.7 
6.3 
4.9 
5.5 
5.7 



5.5 
5.5 



5.8 
5.2 



4.9 

5.5 



5.4 

5.5 



5.4 
5.5 



4.2 
4.3 



5.5 
4.6 
3.6 
4.4 
4.4 
3.6 
3.8 



4.2 
4.2 



4.5 
4.0 



5.7 
4-2 



3.8 
4.3 



4.3 
4.2 



1.3*** 0.000 
1.2*** 0.000 



-0.3 
1.8*** 
1.2*** 
1.9*** 

0.5 

1.9*** 
1.9*** 



0.373 
0.000 
0.001 
0.000 
0.155 
0.000 
0.000 



1.2*** 0.000 
1.3*** 0.000 



1.4*** 0.000 
l.l*** 0.000 



-0.8 0.554 
1.3*** 0.000 



1.5** 0.037 
1.2*** 0.000 



1.1*** 0.000 
1.3*** 0.000 



0.1 0.855 



0.000 



0.0 



0.2 



-2.1 



0.3 



-0.2 



0.913 



0.450 



0.124 



0.695 



0.526 



(continued) 
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TABLE 3.11 (contlnutd) 







Average Number of 
Services Received 

Durina 1987-1988 






Subgroup 




Chiricttrlstic tnd Subgroup 


Sample 
Size 


Experi mentals 


Control s 


Subgroup 
Impact 


r 


Impact 

D1ff*r*net'' 


P 


H«s OMn chlld(rtn) 
Yts 
No 


28 

1.205 


5.4 

5.5 


3.4 

4.3 


1.9** 
1.2*** 


0.043 
0.000 


0.7 


0.473 


Ever In a suMMtr work 
progrm 

Yts 

No 


495 

738 


5.8 
5.3 


4.6 
4.0 


1.3*** 
1.2*** 


0.000 
0.000 


0.0 


0.940 


Employtd «nyt1iiit during 
tht prtvlous school year 

Yts 

No 


384 
849 


5.3 
5.6 


4.2 
4.3 


1.1*** 


0.000 
n nnn 


-0.2 


0.558 


Evtr txpelltd or suspended 
Yts 
No 


227 
1.006 


5.4 
5.5 


4.2 
4.2 


1.2*** 
1.3*** 


0.000 
0.000 


0.0 


0.940 


Ethni c1 tv 
Whitt, non-Hispanic 
Black, non-Hispanic 
Hispanic 
Other 


114 
849 
179 
91 


3.9 
5.8 
5.1 
5.5 


3.8 
4.3 
4.6 
3.5 


0.1 

1.5*** 
0.5 

2.0*** 


0.837 
0.000 
0.185 
0.000 


— 




Faally rtctiving cash 
wtlfart 
Yts 

No 


383 
850 


5.3 
5.6 


4.2 
4.3 


1.2*** 
1 3*** 


0.000 
0 000 

V . WW 


-0.1 


0.674 


Age 
18 or older 
17 or under 


124 
1.109 


5.2 
5.5 


4.5 
4.2 


0.7 
1.3*** 


0.124 
0.000 


-0.6 


0.187 


Courses Rostly college 
preparatory 

Yes 

No 


449 

784 


6.0 
5.2 


4.2 

4.3 


1.8*** 
0.9*** 


0.000 
0.000 


0.9*** 


0.004 


Parent (s) graduated from or 
attended college 

Yes 

No 


405 
828 


5.6 
5.4 


4.2 
4.3 


1.4*** 
1.2*** 


0.000 
0.000 


0.3 


0.376 



(continued) 
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TABLE 3.11 (contlnutd) 







Avtragt Numbtr of 














Strvlcts Rtctlvtd 














Durina 1987-1988 






Subgroup 






Sampla 






Subgroup 




iMpact 




Ch«racttristic and Subgroup 


Sizt 


Exptrlntntalt 


Controls 


Impact 




Dlfftrtnct'* 


P* 


Fanlly rtctlving Food Stamps 












0.1 


0.864 


Yts 


317 


5.4 


4.1 


1.3*** 


o.ooo 


— 


— 


No 


916 


. 5.5 


4.3 


1.2*** 


o.ooo 


— 


— 


Evtr In a ytar-round training 
















program not sponsortd by the 
















sanpit MBbor*s high school 












-0.4 


0.425 


Yts 


100 


5.9 


5.0 


0.9* 


0.087 






No 


1.133 


5.4 


4.2 


1.3*** 


O.ooo 







SOURCES: NORC calculations from Career Beginnings enrollment form and 1988 survey data. 



NOTES: This table Includes services received from Career Beginnings and/or other sources. A 
participant could receive a maximum of 11 services (see Table 3.6). 

Calculations for this table used self -reported data for all 621 experimental s and all 
612 controls Mho responded to both the 1988 and 1989 surveys. Including those with values of zero for 
outcomes and those Mho Mere assigned to Career Beginnings but did not participate. Subgroup sample 
sizes for some characteristics may fall short of the total number of sample members because of Items 
missing from data for some survey responders. 

Average experimental and control group outcomes reported here are adjusted means from 
tMOHifay analysis of covarlance procedures controlling for up to 23 kinds of difference In 
characteristics^ other than the characteristic used to define subgroups, before random assignment. 
The tMO categories used as factors Mere research assignment and, one at a time, the baseline 
characteristics Indicated (see Ostle, 1975, p. 454). Because of rounding, there may be slight 
discrepancies In reported sums and differences of these adjusted means. 

*TM0-ta11ed t-tests Mere applied to m1 thin-subgroup Impacts and also to differences 
betMoen subgroup Impacts. For each characteristic Mith more than two subgroups (site and ethnicity), 
an F test was applied to the Interaction between that characteristic and experimental or control 
status* The columns labeled "p* are the statistical significance levels of each Impect, each 
difference between Impacts, or each F statistic: that Is, p Is the probability that sample estimates 
are non-zero only because of random error. Statistical significance levels are Indicated as 
• 1 percent or less; • 5 percent; ♦ « 10 percent. 

'^For each characteristic that has only two subgroups, the subgroup Impact difference 
Is the Impact within the first subgroup, less the Impact within the second subgroup. 




general rule that being in the experimental group was associated with a signiGcant increase 
in the number of services received. 

But did belonging to any particular subgroup also rf^ult in a significant increase in 
services? Were women, for instance, likely to receive more services than men? Were students 
who had never dropped out likely to receive more services than those who had? The two 
right-most columns of the table answer these questions. The column labeled "Subgroup Impact 
Difference" shows the difference between the impacts registered by subgroups within a category, 
and the "p** column shows the probability that the difference was itself statistically significant. 
The subgroup impact difference for three of the baseline characteristics was statistically 
significant (that is, had an associated p value of 0.10 or less). The fint concerned those 
students who were taking mostly college preparatory courses. Career Beginnings was notably 
more 2;uccessful in increasing the level of service receipt among experimental in college 
preparatory courses vis-a-vis their control group counterparts who were also in a college track 
than in increasing service receipt among non-college-preparatory experimental vis-a-vis non- 
college-preparatory controls. 

Site and ethnicity also showed statistically significent differences, although the table does 
not specify the "difference" per se because multiple categories are involved.^ Table 3.11 yields 
few additional clues about which subgroups benefited most in terms of increased services during 
the program year. 

E. Service Receipt in the Year After Career Beginnings 

If higher proportions of experimental than controls continued to receive career services 
during the post-graduation year, larger impacts might be expected. There are two factors that 
make a hypothesis of increased levels of services to expcrimentals during this period a 
reasonable one. 

Fint, all but two of the seven sites (Jacksonville and Rochester) received supplemental 
funding to provide more sendees to program alumni than were called for in the original model.^ 
Known as "Next Step/ this entailed different activities at the sites. For example, Bronx 



'The value in the *p* column in these cases is the F-tcst value. As with the p values for two-category 
characterLuGS, an F value of less than 0.10 represent a suttstically signiflcant difference among the 
catepries. 

^Although Youngstown Sute University left the national demonstration, the national program office 
contracted with a fonner Career Beginnings staff member at that site to provide these follow-up activities. 
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Commfinity College set up a peer advisor and tutoring network, in which former Career 
Beginners attending other colleges were trained to provide assisunce to recent Career 
Beginnings graduates who were freshmen at their institutions; the Indianapolis site established 
an alumni case manager position. Career Beginnings students contacted and aided in this way 
might be expected to weather the vicissitudes of freshman year more successfully than controls, 
who presumably lacked such assistance. Second, the very experience of receiving assistance and 
support during the Career Beginnings program year may have increased participants' propensity 
to seek help afterwards. 

The second follow-up interview asked former Career Beginnings participants whether 
they had any contact with the program in the year after high school. The majority, as indicated 
in Table 3.12, had not However, the proportion retaining some tie to the program differed 
considerably among the sites. Seventy percent of Gary participants continued to be in touch 
with Career Beginnings staff, compared with fewer than one-quarter of the Rochester 
pakdcipants. 

Table 3.13 indicates the extent to which members of the research sample received 
services of various kinds, from any source, during the 1988-89 follow-up year. For both 
experimental and controls, levels of service receipt were low, with virtually no statistically 
significant differences between the two groups. (Experimental were slightly more likely than 
controls to get counseling.) 

To summarize: At most sites. Career Beginnings delivered on its promise of providing 
more educational and career services to program participants than they would have received 
otherwise. But the premise that these students would otherwise have received few if any 
services was not borne out In faa, service receipt by members of the control group was 
substantial, albeit lower than that of experimental. 

There are at least two explanations for why controls received more services than 
anticipated. First, both the general Career Beginnings eligibility criteria and the specific 
screening methods used by sites to select participants may have served to ensure that most 
members of the research sample - experimental and controls alike - would be motivated to 
seek and make use of available services. A second explanation is that economically 
disadvantaged college-oriented youths may not be as overlooked as had been thought. At most 
sites, there were other programs for disadvantaged teens for which the entry criteria overlapped 
in some measure with those of Career Beginnings. Moreover, the high schools provided some 
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TABLE 3.12 



AMOUNT OF CONTACT WITH CAREER BEGINNINGS DURING FIRST YEAR AFTER HIGH SCHOOL: 
PERCENTAGE DISTRIBUTION OF EXPERIMENTAL GROUP PARTICIPANTS. BY SITE 



Site 


Sample 
Size 


A Lot 
(%) 


Some 


A Little 


None 


All 
Categorl es 


P 


Thft Bronx. New York 


58 




9.4 


2n 7 


AO n 


inn n*** 


n AAA 


GArv. Indiana 


82 


15 9 


25 5 






ion n 




inaianapoiiSt Indiana 


81 


4.9 


19.8 


27.2 


48.2 


100. o*** 


0.000 


Jacksonvlllet Florida 


61 


6.6 


8.2 


18.0 


67.2 


100. o*** 


0.000 


Rochester, New York 


65 


3.1 


7.7 


12.3 


76.9 


100. o*** 


0.000 


Santa An«t California 


72 


4.2 


20.8 


37.5 


37.5 


100. o*** 


0.000 


Youngs town, Ohio 


73 


8.2 


6.9 


17.8 


67.1 


100. 0»»* 


0.000 


All Sites 


492 


7.1 


14.2 


23.6 


55.1 


100. 0*** 


0.000 



SOURCES: NORC calculations from Career Beginnings enrollment form and 1989 survey data. 



NOTES: Calculations for this table used self-reported data for all 494 experimental s Mho responded to both the 1988 and 
1989 surveys and who reported participation In the Career Beginnings program. Sample sizes reported may fall short of this 
number because of Items missing from data for some survey responders. 

Because of rounding, rows may not total exactly 100.0 percent. 

*The column labeled *p* Is the statistical significance level of the differences among category proportions: that 
Is, p Is the probability that observed proportions are different only because of random error. A Pearson chl-squar test was 
used to test the hypothesis of equal proportions. Statistical significance levels are indicated as • 1 percent or less; 
• 5 percent; ♦ • 10 percent. 
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TABLE 3.13 



RECEIPT OF SELECTED SERVICES. 
BY RESEARCH GROUP: 1988-1989 



OutCOM 


Sample 
size 


Experlmtnttls 


Controls 
(%) 


D1 f f erence 


p* 


Recti vtd help with studying and 
test»tek1ng skills 






19.3 


•1.1 


0.593 


Took classes In reading or math, 
apart froii regular school classes 


1,231 






1.5 


0.494 


Got Individualized academic tutoring 


1 ,Z31 


19 ft 


10 1 


2.7 


0.147 


Attended a career fair or otner 
job Information event 


1.230 


15.0 


12.9 


2.1 


0.280 


Attended a job»read1ness skills 
workshop on dress, lateness, etc. 


1.230 


9.4 


8.0 


1.4 


0.401 


Met at least twice with counselor 


1.231 


26.5 


22.2 


4.3* 


0.081 



Sanple Size 



1.233 



621 



612 



SOURCES: MDRC calculations from Career Beginnings enrollment fonn and 1989 survey data. 

NOTES- This table Includes services received from Career Beginnings and/or other sources. 

Calculations for this table used srlf-reported data for all 621 txperlmtntals and 
all 612 controls who responded to both the 1988 and 1989 surveys. Participation In selected 
services was measured by affirmative answers to 1989 survey questions. Sample sizes may .all 
short of the total number of sample members because of items missing from data for some survey 

rcsponders. , . . . 

Because of rounding, there may be slight discrepancies in reported sums and 

differences of the experimental and control proportions. 

•The column labeled "p" Is the statistical significance level of the difference 
between experimental and control proportions: that is. p is the probability that observed 
proportions art different only because of random error. A two-tailed t-test was used to test 
the hypothesis of equal proportions. Statistical significance levels are Indicated as 
• 1 percent or less; ** • 5 percent; • • 10 percent. 
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miitinrr in a number of ireas, from sponsoring college and career fairs to helping students 
prqwre Cor college admissions tests. In any event, as discussed in Chapter 4, the levels of 
service receipt among membeis of both groups are critical to understanding the impacts 
registered by the program. 
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CHAPTER 4 
PROGRAM IMPACTS 



Eighty-six percent of the young people in the Career Beginnings study sample had 
graduated fxom high school by the end of June 1988 and were ready to embark on their 
futures.^ Some had a fairly clear idea of what their futures would hold: They had applied to 
and been accepted for college or had specific work or training alternatives in mind. Others 
were less sure; some of these had not applied to college or had applied but not been 
accepted.^ Their options would unfold over the coming months. 

This chapter uses survey data gathered about one year after high school graduation to 
report on the effectiveness of Career Beginning in attaining its immediate objectives: increasing 
rates of college attendance as well as helping the non-college-bound to And better jobs than 
they would otherwise obtain. In both these ways, the program sought to enhance the long- 
term career potential of its participants. 

To evaluate any program, it is necessary to answer two basic questions. First, on average, 
what happened to those who were offered the program? Second, on average, what wouki have 
happened to them had they not been offered the program? The average effect, or **impact,** 
of a program is the difference between these two outcomes. As explained in Chapter 2, since 
assignment to Career Beginnings was random, there were no overall systematic differences 
between experimentals and conlrols at enrollment, and outcomes for controls provide a measure 
of what would have happened to experimentals without the program. Chapter 3 discussed 
diiTerences in the levels of service receipt of those in the experimental Lnd control groups. 

This chapter addresses several key questions related to the impacts in this service 
difference on pos\ -secondary education, employment, and other aspects cf sample members' 
lives. With respect to education, it asks, first: What impact did the Career Beginnings 
program have on enrollment in post-secondary education - in particular, college? Did Career 



^86.8 percent of the experimenuls and 85.3 percent of the controls had received a high school diploma 
ly that time, a difference that was not statistically significant. During the follow-up year, about 7 percent 
more of those in both groups who did not receive a diploma with their graduating class acquired either 
a hidi school diploma or a GED certificate 

^About IS percent of the sample members who had no. applied or been accepted to college by the 
spring of their senior year went to a two- or four-year college during the follow-up year. 
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Befinning» affect whether students completed their freshman year in college? Moreofver, how 
did the impact on college-going vary by site and by the characteristics of the sample at 
baseline? What kinds of programs did sample members choose? 

The analysis also expbres the short-term relationship between access to the program and 
subsequent employment If Career Beginnings succeeds in increasing college attendance, then 
one might expect those in the experimental group to work and earn less than controls in the 
short run. This is because college can be seen as an investment that entaik the sacrifice of 
immediate earnings in the interest of long-term income gains. The study therefore asb: Did 
Career Beginnings have an impact on the employment and earnings of sample members during 
the year-long follow-up period? Did it affect their likelihood of entering the military? And, 
were experimental more likely than controls to be engaged in some kind of productive activity, 
whether attendance at college, employment, or military service? 

Finally, the research investigates other questions of interest. Did Career Beginnings affect 
sample members' place of residence? Was there an impact on marital status and child-bearing? 

In the main body of the chapter, impacts are presented for all members of the 
experimental and control groups. The last section of the chapter discusses the adjustments 
that can be made if the impacts are reported only for experimental who actually participated 
in Career Beginnings, or if the impacts of Career Beginnings on controls who recent services 
from the program are taken into account. It should be noted, however, that these adjustments 
are simpty another way of looking at the same results and would not materially change the 
story that emerges from the data. 

L Impacts on College-Going and Type of Educational Institution Attended 

The impact analysis reveals that, in keeping with the objectives of Career Beginnings, 
more experimental than controls attended college during the follow-up period. The first line 
of Table 4.1 shows that 4&5 percent of the controls attended a four-year or two-year college 
at some point between June 1988 and May 1989, while 53.2 percent of experimentals did so. 
This impact of 4.7 percentage points represents a 9.7 percent increase over the control group 
mean and was statistically significant, that is, unlikely to have arisen entirely by chance. 

The findings indicate that almost half of the control group attended college, even without 
substantial access to Career Beginnings. As pointed out in Chapter 1, in recent years 
approximately 58 percent of all high school graduates have gone on to colk^^ge by the age of 



TABLE 4.1 

IMPACTS ON POST-SECONDARY EDUCATION: JUNE 1988-NAY 1989 



OUtCOM 


Exptrlmtntals 


Controls 
(*) 


Olfftrtnct 


P* 


Ptrctntagt 
Changt^ 


Evtr «tttnd«d « 4<« or Z-ytar colltgt 
AttiMtd A #«yttr coiitgt 
Atttndtd a 2-yt«r colltgt 


S3.2 
39.0 
18.3 


48.5 

33.9 
16.2 


4.7* 

Z.Q 

2.1 


0.071 
0.397 
0.290 


9.7 

6.0 
13.0 


Evtr ttttndtd t vocational school 


8.3 


12.2 


-3.9** 


0.022 


-32.0 


Evtr atttndtd any font of 
pott-stcondary tducation 


61.2 


59.4 


1.8 


0.491 


3.0 


Evtr rtctlvtd a scholarship 


39.3 


36.0 


3.3 


0.210 


9.2 


Evtr took out a loan for school 


21.6 


21.5 


0.1 


0.971 


0.5 


Sanpit Slit 


621 


612 









SOURCES: NORC calculations from Carter Beginnings enrollment form and 1989 survey data. 



NOTES: Calculations for this table used self-reported data for all 621 experimental s and 
all 612 controls who responded to both the 1968 and 1989 surveys, Including those with values of 
lero for outcomes and those who were assigned to Career Beginnings but did not participate. 

Average experimental and cot^trol group outcomes reported here are adjusted means 
from a linear analysis of covarlance procedure controlling for 24 kinds of difference In 
characteristics before random assignment (see Ostle, 1975, p. 461; Cave, 1987). Because of 
rounding, there may be slight discrepancies In reported sums and differences of these adjusted 
means. 

*The column labeled "p" Is the statistical significance level of the difference 
between experimental and control group outcomes: that Is, p Is the probability that average 
outcomes art different only because of random error. A two-tailed t-test was applied to 
differtncts bttwttn avtragt txptrlmtntal and control group outcomts. Statistical significance 
levels are Indicated as *** > 1 percent or less; ** « 5 percent; * > 10 percent. 

t»Tht percentage change column expresses the difference as a percentage of the 
control group outcome. 



24; thuSt it is noteworthy that a group of students from disadvantaged communities who were 
high school juniors at the onset of the study achieved a 48.5 percent rate of college attendance 
one year after their expected completion of high school. 

Slightly more than one-third of the controls attended four*year colleges, compared to 35.6 
percent of experimental, a difference of 2 percentage points. Slightly more than 16 percent 
of controls and 18.3 percent of experimentals attended a two-year college, an impact of 2.1 
percentage points. Neither impact was statistically significant 

Just over 12 percent of controls, compared to about 8 percent of the experimentals, 
attended vocational schools, a reduction of 3.9 percentage points that was statistically 
significant The table further indicates that when all forms of post*secondary education - 
four-year colleges, two-year colleges, and vocational schools - are considered together, 59A 
percent of controls and 61.2 percent of experimentals attended one or more of these 
institutions by May 1989, a difference of 1.8 percentage points that was not statistically 
significant Career Beginnings did not, therefore, lead to an overall increase in post-secondary 
education. 

The data suggest, however, that the program induced some experimentals to change the 
kind of post*secondary schooling they would otherwise have received. Apparently, Career 
Beginnings may have led some who would have attended vocational programs to choose two- 
year degree programs or four*year colleges, and led others who would have begun at a two* 
year college to go into bachelor's degree programs. 

Interestingly. Career Beginners were not better financed to attend college than their 
control group counterparts. Thirty-nine percent of the experimentals reported having received 
a scholarship compared with 36 percent of the controls, a difference that was not statistically 
significant Another way of viewing these findings is that about three-quarters of the college 
attenders in both groups received scholarship aid.^ Just over one-fifth of all sample members 
- experimentals and controls alike - took out a loan to attend school. 



^That is, S3.2 percent of all experimentals ever attended college, and 39.3 percent ever received a 
scholarship; 393/53.2 > 73.8 percent. Among controls, 48.5 percent attended college, and 36.0 percent 
receded a scholarship; 36.0/48.5 « 74.2 percent. 
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n. ImMda OB Length of Stay and Month-bv-Month Attendance in Eduaitiomil iMtltiittoBg 

Table 42 shows length of stay and month-by-month attendance at four-year and two-year 
colleges and vocational schools. The table includes all experimental and controls, both those 
who entered post-secondary education and those who did not. 

The top panel shows that Career Beginnings increased the length of stay at four-year 
and two-year colleges, considered together, by a little over one-half month, an impact that was 
statistically significant/ This finding reflects two facts: that experimental were more likely 
to go to college in the first place and that, as the next panels of the table make clear, they 
were also more likely to begin four-year colleges "on schedule** in the fall semester rather than 
later on. The month-by-month impact on attendance at two- or four-year colleges grew from 
4.7 percentage points in August to about 6 percentage points in September, October, and 
November, as also shown in Figure 4.1. Thereafter, the experimental -control differential began 
to narrow, as some additional controls enrolled in college during the spring term. 

As noted in Table 4.1, experimentals were significantly less likely than controls to enroll 
in vocational schools, and this is reflected in their shorter length of stay and lower month-by- 
month attendance rates in these institutions. 

m. staving In College 

An important gauge of success in college is the extent to which students who begin 
college remain there and eventually graduate. Because of the brevity of the follow-up period, 
this evaluation is necessarily limited in what it can say about this. However, completion of a 
year of college is a preliminary indicator of longer-term success. 

The evaluation takes two different approaches to measuring retention in college. One 
is to note the proportion of students who ever attended college and were still attending in 
May 1969. Table 4.2 shows that during that month, 47.9 percent of the experimentals and 
43.4 percent of the controls were attending two-year or four-year colleges, for a statistically 
significant impact of 4.5 percentage points. Thus, the experimental-control differential in 
college attendance that was evident at the beginning of the school year held up at its end. 



^Note that these flgures include youths in each group who did not attend college and hence stayed 
for zero months. 
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TABLE 4.2 

IMPACTS ON EDUCATIONAL LENGTH OF STAY AND ON MONTHLY EDUCATIONAL STATUS 



OutcQM ExptrlMnt«1s Controls Difftrence 



Lt/igth of st«y through Nay 1989 
(In Months) «t « 



4-yMr colltgt 


3.25 


2.94 


0.31 


0.168 


2«yMr colltgt 


1.46 


1.31 


0.14 


0.402 


4- or 2-yt«r colltge 


4.64 


4.13 


0.51** 


0.033 


Vocttlonti school 


0.51 


0.83 


-0.32** 


0.012 


Colltgt or vocational school 


5.14 


4.85 


0.29 


0.229 



Atttndtd « 4- 

or 2-yt«r college during (%) 



«#unv ivoo 


D.D 


A 1 

4.1 




o.zzz 


July 


& 7 
D. / 


D.D 


1.1 


A Ana 






91 9 

ClmC 


1 7** 
4./ 


A A9C 

0.03d 




40. / 


JA 7 

40./ 


5.8** 


A AOI 

O.OZl 






in 0 

4U.3I 


A 1** 
0.1 


A A17 
0.01/ 


November 


46.7 


40.8 


5.9** 


0.021 


DeceMbtr 


46.1 


40.3 


5.7** 


0.026 


Jenuery 1989 


47.9 


43.5 


4.4* 


0.095 


February 


48.0 


44.2 


3.8 


0.146 


March 


47.8 


44.5 


3.4 


0.197 


April 


47.5 


43.5 


3.9 


0.132 


May 


47.9 


43.4 


4.5* 


0.081 


ttended a 4^year 










allege during (%) 










June 1988 


4.5 


3.0 


1.5 


0.145 


July 


5.2 


3.9 


1.3 


0.283 


August 


17.5 


14.8 


2.7 


0.157 


Septenber 


32.9 


28.7 


4.2* 


0.080 


October 


33.2 


28.9 


4.3* 


0.072 


Novenber 


33.0 


28.9 


4.1* 


0.085 


Otce«ber 


32.5 


28.8 


3.7 


0.116 


January 1989 


32.9 


30.7 


2.3 


0.344 


February 


33.2 


31.0 


2.3 


0.345 


March 


33.3 


31.6 


1.6 


0.493 


April 


33.2 


31.5 


1.7 


0.476 


May 


33.3 


31.8 


1.5 


0.530 



(continued) 



100 

-64- 



TABLE 4.2 (continued) 



Outcont 


Experimental s 


Controls 


01 f f ertnce 


^a 

P 


Atttndtd « 2-year 










colltgt dHrIng (%) 










Junt 1988 


1.1 


1.5 


-0.4 


0.557 


July 


1.6 


2.0 


-0.4 


0.559 


August 


8.9 


7.5 


1.3 


0.363 


Sftptanbtr 


14.5 


13.2 


1.2 


0.502 


October 


14.6 


13.3 


1.3 


0.468 


NovMbtr 


14.4 


13.1 


1.3 


0.471 




14.3 


12.8 


1.5 


0.417 


January 1989 


15.7 


14.0 


1.8 


0.350 


Ftbruary 


15.4 


14.2 


1.2 


0.525 


March 


15.1 


13.9 


1.2 


0.510 


April 


14.8 


13.1 


1.7 


0.347 


Nay 


15.3 


12.8 


2.5 


0.177 


Atttnded a vocational 










school during (%) 










Junt 1988 


0.8 


1.7 


-0.9 


0.161 


July 


1.6 


2.7 


-1.1 


0.192 


August 


2.3 


4.2 


-1.9* 


0.060 


StptiMbtr 


4.2 


7.9 


-3.7*** 


0.006 


October 


4.3 


O.Z 




n AAA 


NovMber 


4.8 


8.2 




0.012 


DacMiber 




7 Q 


" J. O 


0 008 


January 1989 


5.4 


8.6 


-3.2** 


0.027 


February 


5.6 


8.4 


-2.8* 


0.057 


Narch 


5.9 


8.7 


-2.8* 


0.060 


April 


6.4 


8.4 


-1.9 


0.191 


May 


5.6 


8.2 


-2.6* 


0.071 


Sample Size 


621 


612 







SOURCES: MORC calculations from Career Beginnings enrollment form and 1989 survey 

data. 



NOTES: Calculations for this table used self-reported data for all 621 
exparlmtntals and ull 612 controls who responded to both the 1988 and 1989 surveys. 
Including thc$e trith values of zero for outcomes and those who were assigned to Career 
Beginnings but did not participate. 

Average mptrlmental and control group outcomes reported here are 
adjusted means from llnMr analysis of covarlance procedure controlling for 24 kinds of 
difference In characteristics before random assignment (see Ostle, 1975, p. 461; Cave, 
1987). Because of rounding, there may be slight discrepancies In reported sums and 
differences of these adjusted means. 

*The column labeled "p" Is the statistical significance level of the 
difference between axperlmental and control group outcomes: that Is, p Is the 
probability that averaga outcomes are different only because of random error. A 
fwo-talled t-test was applied to differences between average experimental and control 
group outcomes. Statistical significance levels are Indicated as > 1 percent or 
less; ■ 5 percent; ♦ ■ 10 percent. 



FIGURE 4.1 

MONTHLY ATTENDANCE AT TWO- OR FOUR- YEAR 
COLLEGES. BY EXPERIMENTALS AND CONTROLS 



Percentoge of Group Attending 
60 



I 

ON 

I 
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Experimentals 



Controls 



Month 



1G3 



SOURCE: Table 4.2. 
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A second approach to examining retention is adopted in Table 43, which reports the 
proportion of sample members who attended post-secondary institutions of different types in 
October 1988 and remained continuously enrolled through May 1989. Because the percenUges 
are based on college and vocational school attenders, rather than on all experimentals and all 
controls, the table does not show the impact of assignment to the experimenul group, and is 
not labeled as such. 

The table shows that equal percentages of experimentals and controls - about 90 percent 
for both groups - who were enrolled in two- or four-year colleges in October were still 
attending in May. In other words, the large majority of individuals who began college in the 
fall remained there to complete their fint year. There were differences, however, according 
to the type of educational institution attended, with those in four-year colleges more likely to 
remain enrolled than those in two-year colleges. This finding is likely in part to reflect 
differences in the characteristics of students attending these institutions. 

Lower percentages of sample memben (56.5 percent of the experimentals and 65.7 percent 
of the controls) who attended vocational schools in October remained there continuously 
through May. Some fall-off may have occurred as sample members completed vocational 
programs lasting less than eight months; some of these young people, too, may have enrolled 
in two- or four-year colleges later in the academic year. 

IV. Site and Subgroup Variations In Collcge^Going 

The primary goal of the Career Beginnings impact evaluation is to estimate overall impacts 
on college-going, and the sample was large enough for that purpose. Although the sample 
provided considerably less statistical power for estimating impacts on different subgroups, a 
thorough analysis was carried out for one important outcome, attendance at a four-year or two- 
year college during the year after high school graduation. 

Table 4.4 displays the results. The table resembles Table 3.11 and, like that table, 
partitbns the sample into subgroups defined by site and by other key characteristics of sample 
members at baseline.^ 



^Impacts are examined wahin subgroups while adjusting sutistically for measured differences across 
subgroups. These adjustments are important because, as noted in Chapter 2, characteristia of program 
eligibles varied considerably among the sites, and because, owing to small sample sizes, random assignment 
did not necessarily ensure completely equivalent experimental and control groups within any given site. 
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TABLE 4.3 



PERCENTAGE OF THOSE AnENDING AN EDUCATIONAL INSTITUTION IN OCTOBER 1988 
WHO WERE STILL AHENDING IN NAY 1989 



Sanplt ExptrlMntals Controls 



Typt of Pott-Stcondary Institution 


Sizt 


(%) 


(%) 


Olfftranca 


P* 


4- or 2-yMr collogt 


543 


90.2 


90.7 


-0.4 


0.861 


4-yMr colltgt 


383 


93.4 


94.7 


-1.3 


0.588 


2-yMr colltgt 


172 


83.9 


82.2 


1.7 


0.778 


Vocational school 


77 


56.5 


65.7 


-9.2 


0.478 



SOURCES: NORC c«1cu1«t1ons from C«rttr Btglnnlngs tnrolliiitnt fon» «nd 1989 survty data. 

NOTES: Calculations for this tabit ustd self-rtporttd data for all 621 txptrlMntals 
and all 612 controls Mho rtspondad to both tht 1988 and 1989 survtys. Including thosa with 
valuts of laro for outccmts and thost who wart asslgnad to Cartar Beginnings but did not 
partlclpata* Each Una of tha tabia usad a diffartnt %mpU. In Una 1. data wara usad for 
sanpla Mubars attending a four- or two-yaar collage In October 1988; In line 2. for those 
attending a four-year college In October 1988; In line 3. for those attending a two-year 
college In October 1988; and In line 4. for those attending a vocational school In October 
1988o 

Average experimental and control group outcomes reported here are adjusted 
means from a linear analysis of covarlance procedure controlling for 24 kinds of difference 
In characteristics before random assignment (see Ostle, 1975. p. 461; Cave, 1987). Because 
of rounding, there may be slight discrepancies In reported sums and differences of these 
adjusted means. 

*The column labeled ""p*" Is the statistical significance level of the 
difference between experimental and control group outcomes: that Is, p Is the probability 
that average outcomes are different only because of random error. A two-tailed t-test was 
applied to differences between average experimental and controi ^roup outcomes. Statistical 
significance levels are Indicated as » 1 percent or less; » 5 percent; * » 10 percent. 
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TABLE 4.4 



IMPACTS ON C0LLE6E AHENOANCE. 
BY SELECTED BASELINE CHARACTERISTICS 







Evtr Atttndtd a 2- 














or 4-Ytar Colltgt, 














^ynt im-mv m 






Subgroup 






Samplt 


ExptrlMtntals 


Controls 


Subgroup 




I«pact 




Characltrlttlc and Subgroup 


Sizt 


(*) 


(*) 


Iispact 




01 f f trtnct° 




Stx 












5.9 


0.273 


FfMit 


791 


54.7 


47.9 


6.8** 


0.036 


— 


— 


Naif 


442 


50.2 


49.4 


0.9 


0.843 


— — — 


— — — 


Sitt 












— 


0.485 


Tht Ironx, Ntw York 


170 


43.3 


45.8 


-2.5 


0.715 




... 


Gary, Indiana 


179 


62.9 


48.7 


14.3** 


0.035 


•-- 


... 


Indlanapollti Indiana 


212 


37.7 


36.5 


1.2 


0.851 


mmm 


... 


JAektMv111#. Florldft 


153 


58.9 


48.4 


10.5 


0.157 






Rochtsttr, Now York 


174 


53.7 


56.7 


-3.0 


0.666 


... 





Santa Ana, California 


162 


67.8 


61.9 


S.9 


0.410 


... 


... 


YoungttOMn, Ohio 


183 


52.1 


44.6 


7.S 


0.270 


... 


... 


Biployid anytiM during tht 
















ytar btfort random asslgnmant 












2.2 


0.668 


Yts 


646 


55.7 


50.0 


S.7 


0.109 


... 


... 


No 


587 


50.3 


46.8 


3.S 


0.349 


... 


... 


Rtctnt gradt avtragt 












0.9 


0.857 


A or B 


595 


63.4 


58.3 


S.2 


0.167 


... 


... 


C, D, or F 


638 


43.6 


39.4 


4.2 


0.243 


... 


... 


Evtr a school dropout 
















for a taaatttr or Boro 












8.1 


0.739 


Yts 


14 


33.8 


21.1 


12.7 


0.601 






No 


1,219 


53.4 


48.8 


4.6* 


0.079 


... 


... 


















anothtr languagt It usually 
















tpokan at hoM 












8.6 


0.528 


Yts 


46 


54.4 


41.4 


12.9 


0.333 






No 


1.187 


53.1 


48.8 


4.4* 


0.099 






Living iflth both partnts 












-1.2 


0.819 


Yts 


528 


53.6 


49.7 


4.0 


0.314 






No 


70S 


52.8 


47.6 


5.2 


0.129 







(continued) 
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TABLE 4.4 (contlnutd) 







Evtr Atttndtd « 2- 
or 4-Yt«r Colltgt, 
Junt 19884(av 1989 






Subgroup 

li^tCt 

Difftrtnct'' 

VIII W WIIWV 




Chartcttrlttlc and Subgroup 


Samplt 
Sizt 


Exptrlmtnttls 

(%) 


Controls 
(%) 


Subgroup 
Inptct 


r 


r 


Hat om chlldirtn) 
Ytt 
No 


28 
1.205 


49.6 

53.3 


36.1 
48.8 


13.5 
4.5* 


0.435 
0.088 


9.1 


0.60S 

mmm 


Evtr In « suMitr work 
progrM 

Ytt 

No 


495 

738 


48.3 
56.4 


44.2 
51.4 


4.2 

5.0 


0.308 
0.134 


-0.9 


0.869 


Etaploytd anytlmt during 
tht prtvlout school yt«r 

Ytt 

No 


384 


47.3 
55.8 


45.5 
49.9 


1.8 


0.702 


-4.2 


0.455 


Evtr txptlltd or susptnded 
Ytt 
No 


227 
1,006 


38.9 
56.4 


37.8 
50.9 


1.1 

5.5* 


0.854 
0.056 


-4.4 


0.512 


Ethnicity 
Whitt, non-Hi tptnic 
Black, non-Hi spanic 
Hitptnic 
Othtr 


114 
849 
179 
91 


44.3 

53.1 
51.3 
66.8 


46.7 
48.1 
50.6 
49.2 


-2.4 
5.0 
0.7 

17.6 


0.776 
0.108 
0.913 
0.065 


mmm 


U.413 

mmm 


Fanlly rtctlving cash 
wtlfart 

Ytt 

No 


383 
850 


51.8 
53.8 


49.0 
4«.3 


2.8 
S S* 


0.547 
0 n7fl 


-2.7 


0.626 


Ago 
18 or oldtr 
17 or undtr 


124 
1.109 


53.0 
53.2 


49.3 
48.4 


3.7 
4.8* 


0.649 
0.081 


-1.1 

MM 

mmm 


0.902 

mmm 
mmm 


Courttt Mottly colltgt 
prtparatory 

Ytt 

No 


449 

784 


66.8 
45.4 


61.4 
41.1 


5.4 
4.3 


0.209 
0.193 


1.2 


0.832 


Partnt(t) graduattd from or 
atttndtd colltgt 

Ytt 

No 


405 
828 


59.1 
50.3 


52.1 
46.8 


7.0 
3.5 


0.124 
0.262 


3.4 


0.538 



(contlnutd) 
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TABLE 4.4 (contlnutd) 







Evtr AtttndcJ a 2- 














or 4-Yaar Collcga, 




















Subgroup 






Sanpla 


ExptrlntntaU 


Controls 


Subgroup 




Inpa'^t 




Ch«racttr1st1c «nd Subgroup 


Sizt 


(*) 


(*) 


Impact 


P* 


Difftrtnet^ 


P« 


FMrtly I*tct1v1ng Food Straps 












•0.2 


0.978 




317 


50.5 


46.0 


4.6 


0.373 


... 




No 


916 


54.1 


49.4 


4.7 


0.118 


... 


mmm 


Evtr In a ytar-round training 
















prograM not sponstortd by tht 
















SMpI* Mibtr's high school 












9.1 


0.340 


Yta 


100 


63.6 


50.6 


13.1 


0.155 






No 


1.133 


52.3 


48.4 


3.9 


0.144 







SOURCES: NDRC calculations from Career Beginnings enrol iMnt form and 1989 survey data. 



NOTESt Calculations for this table usi&d self -reported data for all 621 experimental s and all 
612 controls Mho responded to both the 1988 and 1989 surveys, Including those with values of zero for 
outcomes and those Mho were assigned to Career Beginnings but did not participate. Subgroup sample 
sizes for some characteristics may fall short of the total number of samplf members because of Items 
hissing from data for s>me survey responders. 

Average experimental and control group outcomes reported here are adjusted means from 
tro«way analysis of covarlance procedures controlling for up to 23 Mnds of difference in 
-characteristics, other than th% characteristic used to define subgroups, before random assignment. 
The two categories used as factors were research assignment and, one at a time, the baseline 
chaibmri sties Indicated (see Ostle, 1975, p. 454). Because of rounding, there may be slight 
discrepancies In reported sums and differences of these adjusted means. 

*TMo-ta11ed t-tests were applied to w1 thin-subgroup Impacts and also to differences 
between subgroup Impacts. For each characteristic with more than two subgroups (site and ethnicity), 
en F test was applied to the Interaction between that characteristic and experimental or control 
status. The columns labeled "p" are the statistical significance levels of each Impact, each 
difference between Impacts, or each F statistic: that Is, p Is the probability that sample estimates 
are non-zero only because of random error. Statistical significance levels are indicated as 
• 1 percent or ^ess:» ** » 5 percent; * « 10 percent. 

^For each characteristic that has only two subgroups, the subgroup Impact difference 
is the impact within the first subgroup, less the impact within the second subgroup. 
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As with Table 3.11, Table 4.4 presents the results of two types of ana^is. It reports, 
Gist, on ¥^ther experimental-control differences are significant wiihin a given subgroup. 
(This difference is shown in the column labeled "Subgroup Impact/ and in the column to its 
right, which indicates the probability that the subgroup impact could have arisen by chance.) 
Because of the small sizes of these subgroups, only very sizable impacts are likely to prove 
sUtistically significant; nonetheless, patterns of impacts are interesting, even when these impacts 
do not attain statistical significance. 

F6r example, the table points to a good deal of variation in impacts by site. The Career 
Beginnings program in Gary had an impact on college attendance of 143 percentage points, 
or 29 percent above the control group attendance rate; this was the only site where the impact 
was sUtistically significant Three other programs had impacu larger than the overall sample, 
but owing to their smaller sample size, none of these impacts was significant The Jacksonville 
program had an impact of 10.5 percentage points, while the Young^town and Santa Ana sites 
registered impacts of 7.5 percentage points and 5.9 percentage points, respectively. At the 
other sites, the impact was smaller 1.2 percentage points in Indianapolis, minus 2.5 percentage 
points in the Bronx, and minus 3.0 percentage points in Rochester. The p value reported in 
the final column of the table indicates that there is a 49 percent chance these differences in 
site impacts could have arisen by chance in drawing a sample of this size from programs with 
equal impacts. 

Site variation in impacts resulted from variations in the rates of college attendance by 
both experimentals and controls. Among experimental the percentage attending college 
ranged from 37.7 percent in Indianapolis to 67.8 percent in Santa Ana. The range in college 
attendance for controls was only slightly smaller: from 36.5 percent in Indianapolis to 61.9 
p<pn:ent in Santa Ana. 

It is striking that the Gary and Jacksonville sites, which were judged to have implemented 
the program most successfully and which showed the greatest experimental-control differential 
in terms of service receipt, also registered the strongest impacts on college attendance. 
Conversely, at the Bronx and Rochester sites, where implementation and service delivery were 
least successful, escperimentals* college attendance did not rise above that of controls. 

The table displays other subgroups for whom Career Beginnings produced differences in 
rates of college-going by experimental and controls that were statistically significant The 
program increased college atiendance for women. It also registered positive impacts for those 
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who had never been expelleu or suspended, had not been employed during the school year 
before Career Beginnings began, and did not come from families receiving cash welfare. These 
data suggest that youny, people with the greatest educational or economic barriers to college 
attendance were not especially helped by the program.^ 

However, caution about such a conclusion is indicated by the results of a second kind of 
analysis used to examine the subgroup issue. This type of analysis reports (in the column 
labeled "Subgroup Impact Difference** and the probability column immediately following) 
\^ther impacts are signiflcantl; ^'iferent for cne subgroup of a given characteristic than 
another - whether, for example, the program proved more effective for women than for men, 
more effective for those who had never dropped out or been expelled or suspended than for 
those who had, and so on. This kind of analysis is particularly important in informing decisions 
about whether the program should target particular kinds of individuate to achieve maximum 
results. 

The table shows that there were no statistically significant differences among the subgroups 
for any of the characteristics indicated. While the difference in impacts by sex (an 
experimental-control difference of 6.8 percentage points for the women and of 0.9 percentage 
points for the men) is not statistically significant, it is worthy of note. No other subgroup 
difference appears to be of comparable importance. 

Efforts were also made to examine program effectiveness for particular clusters of 
subgroups. (For example, experimental-control differences were measured for yotmg women 
whose parents had not attended college and whose families were receiving welfare and for 
young women with a parent who had attended college and whose families were not on welfare.) 
These subgroup Cijster analyses also failed to suggest any particular group which especially 
beneGted from Career Bef*'nnings. 

Appendix Table B.l resembles Table 4.4, but examines subgroup effects on college 
retention rather than on attendance at any time during the follow-up period. (The outcome 
examined is attendance at a two- or four-year college during May 1989.) The results 
correspond in large measure with those reported in the subgroup analysis in Table 4.4, in part, 
no doubt, because those more likely to remain in college are also more likely to have gone 




^However, in some cases (dropouts, students who had problems with English, and students who were 
parents), disadvantaged subgroups with very smali sample sizes showed large - but statistically insignificant 
- Unpacts. 
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there in the Gnt plac^ One difference is notabh, however WhUe the esqiierimental-contiol 
differential in the percentage ever attending college was marked for women (and nonexistent 
for men), there were no statistically significant impacts on retention for either sejc, and 
essentially no differrnce between men and women in attendance during the month of May. 
This suggests that, while the program helped women get into college, it succeeded less well in 
keq>ing them there. 

V. Fkldi of Stiidv 

The 744 sample members Who went on to some form of post-secondary education were 
asked about their planned course of study, and information was obtained for 619, or 83 
percent, of them. (Of course, the students, especially those in four-year colleges, might well 
diange their minds later in their college careers.) About two-thirds of those who continued 
their educations chose fields that could be expected to offer direct preparation for future jobs. 
F6r example, 176 chose business-related courses of study, 56 were pursuing 
oomputer/informatbn science courses, 49 were studying engineering, and 66 were in health 
occupatiCiis or health science areas. Just over one-quarter (26.2 percent) chose liberal arts 
programs. 

VL Short'Tenn Impacts on Emptovment 

Career Beginnings was intended to help participants enter college and to assist them in 
getting better jobs than they would otherwise obtain - jobs with career ladders and higher 
wages. This section examines program impacts on sample members* employment (including 
empk)yment in the militaiy) within the 12 months after their senior year in high school 
Basically, the data confirm that the anticipated short-term trade-off between attending college 
and woridng occurred: While experimentak were more likely than youths in the control group 
to invest in their futures by enrolling in college, they were less likely to be employed, and 
consequently, as a group, their earnings were lower. 

Table 4<5 shows civilian empbyment r^tes for both groups during the follow-up period 
as a whole and during each month. Large majorities in both groups - 79.9 percent of the 
experimentals and 84.1 percent of the controls - worked during this period; the 42 percentage 
point difference between the groups was statistically significant. The month-by-month 
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TABLE 4.5 
IMPACTS ON EMPLOYMENT 



OutCOM 


ExDArlMntal t 






P 


Evtr Miploytd, Junt 198e-my 1989 (%) 


79.9 


84.1 


-4.2* 


0.054 


Ihnbtr of nonths Mploytd, 










JUM 19884Uy 1989 


6.20 


6.48 


-0.28 


0.271 


Ninbtr of Mtks Mploytd, 










JuM 1988-Nay 1989 


24.4 


25.5 


-1.1 


0.327 


Numbtr of hours tmployed, 










Junt 1988-Nay 1989 


775.40 


803.13 


-27.73 


0.488 


Enploytd during (%) 










Junt 1988 


34.6 


41.1 


-6.5** 


0.017 


July 


41.7 


46.7 


-5.0* 


0.077 


August 


46.1 


50.3 


-4.2 


0.137 


SoptMbtr 


46.9 


50.1 


-3.2 


0.252 


Octobtr 


48.9 


52.1 


-3.2 


0.259 






CA 7 


-9 0 
•fc .J 




DocMbtr 


55.5 


55.7 


-0.2 


0.940 


January 1989 


55.9 


58.3 


-2.4 


0.389 


Ftbruary 


57.6 


57.7 


0.0 


0.990 


March 


59.9 


58.4 


1.5 


0.594 


April 


59.9 


59.4 


0.5 


0.862 


Nay 


61.4 


63.3 


-1.9 


0.480 


Sanpla Siza 


621 


612 







SOURCES: NORC calculations from Caraar Baglnnlngs anrollmtnt form and 1989 
survay data. 

NOTES: Calculations for this tabia usad salf-raportad data for all 621 
axparlaantals and all 612 controls Mho raspondad to both tha 1988 and 1989 survays, 
Including thosa with valuas of zaro for outconas and thosa who wara asslgnad to 
Cartar laglnnlngs but did not parti cipata. 

Avaraga axparlaantal and control group outcoMS raportad hara ara 
adjustad naans fron a llnaar analysis of covarlanca procadura controlling for 24 
kinds of diffaranca In charactarl sties bafora random asslgnnant (saa Ostla. 1975, p. 
461; Cava, 1987). Bacausa of rounding, thara may ba slight discrapanclas In raportad 
sums and diffarancas of thasa adjustad maans. 

*Tha column labalad "p" Is tha statistical signlflcanca laval of tha 
diffaranca batwaan axparlmantal and control group outcomas: that It, p Is tha 
probability that avaraga outcomas ara d1 ff arant only bacausa of random •rror. A 
tifO«ta11ad t-tast was appllad to diffarancas batwaan avaraga axparlmantal and control 
group outcomas. Statistical signlflcanca lavals ara Indlcatad u « 1 parcant or 
lass; ^ « 5 parcant; * » 10 parcant. 
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employmcQt ntes, also shown in Figure 4.2, indicate that this difference was driven by the 
fact that controk were more likely than experimentak to be employed during the summer and 
fall after graduation.^ Employment rates for both groups climbed over time, ^^e the 
difference between the groups narrowed substantially and was no longer statistically significant 
by midway through the follow-up period. 

Higher rates of employment for controls than for experimental translated into $216 more 
in earnings over the follow-up period ($3,750.95 for controls versus $3^34,93 for 
e3q)erimentals), a 6.1 percent difference that was not statistically significant (see Table 4.6). 
As would be expected, earnings of both experimental and controls increased over time, as 
shown in Egure 43, as more members of both research groups secured employment Mirroring 
the narrowing differential in employment rates between the two groups, the earnings 
differential also disappeared over time. Experimentals did not fmd better-paying jobs than 
controls, at least as indicated by wage level; for both groups, the average wage during the 
follow-up period was about $4.50 an hour. 

Military service was an alternative to college-going and to civilian employment for a 
relatively small number of teens in both groups: 23 percent of the experimentals and 1.9 
percent of the controls. 

VII, Propram Impacts on "Productive Activity" 

Because a possible trade-off between post-secondary education and work was anticipated, 
the evaluation also undertook to answer a question that assumed both these activities (along 
with military service) were worthwhile and then asked: What proportion of experimentals and 
controls were engaged in at least one of these three kinds of productive activity during the 
follow-up? Table 4.7 displays the results of this analysis. 

The large majority of sample members - 95 percent of experimentals and controls alike 
- were invoWed in a productive activity at some point during the follow-up. "Productive 
activity rates" were similar for the two groups on a month-by-month basis as well. These 
monthly data, in conjunction with earlier findings, suggest that, as would be expected, 
employment was the main form of productive activity in the months immediately following high 



^As Table 43 indicated, experimentals were signincantly more likely than controk to attend college 
during the bit months. 
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FIGURE 4.2 
MONTHLY EMPLOYMENT RATES 
FOR EXPERIMENTALS AND CONTROLS 
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TABLE 4.6 
IMPACTS ON EARNINGS 





Exptrlntntals 


Controls 








it) 


($) 


Sifftrcnci 




Total tamlngs. Junt 1969 


3,534.93 


3,750.95 


-216.02 


0.292 


Earnings during 










Junt 1968 


174.09 


202.13 


-28.04* 


0.088 


July 


243.80 


277.35 


-33.55* 


0.093 


August 


258.21 


287.73 


-29.52 


0.146 


Scptffibar 


Z49 . 72 


282.14 


-3Z.4Z 


A 1 AC 
0.109 


Octobtr 


277.42 


312.77 


-35.35* 


0.093 


NovMbtr 


287.72 


318.19 


-30.47 


0.137 


DtcMbtr 


312.83 


331.06 


-18.22 


0.387 


January 1989 


323.55 


331.78 


-8.22 


0.698 


Ftbniary 


305.13 


311.18 


-6.05 


0.759 


March 


355.91 


357.13 


-1.22 


0.956 


April 


359.60 


353.93 


5.67 


0.802 


Nay 


386.94 


385.57 


1.37 


0.957 


Smplt Size 


621 


612 







SOURCES: MORC calculations from Carter Btglnnlngs tnrolliitnt form and 1989 
survty data. 

NOTES: Calculations for this tabit ustd stif-rtporttd data for all 621 
txptrlntntals and all 612 controls who rtspondtd to both tht 1966 and 1969 survtys. 
Including thost with valuts of ztro for outcomts and thost who wtrt asslgntd to 
Carttr Itglnnlngs but did not partlclpatt. 

Avtragt txptrlntntal and control group outcomts rtporttd htrt art 
adjusttd Mans fron a llntar analysis of covarlanct proctdurt controlling for 24 
kinds of difftrtnct In characttrl sties btfort random asslgnMnt (stt Ostlt. 1975. p. 
461; Cava. 1967). Btcaust of rounding, thtrt may bt slight discrtpanclts In rtporttd 
sums atid difftrtncts of thtst adjusttd mtans. 

*Tht column labtltd *p* Is tht statistical signlficanct Itvtl of tht 
difftrtnct bttwttn txptrlmtntal and control group outcomts: that Is. p Is tht 
probability that avtragt outcomts art difftrtnt only btcaust of random trror. A 
two-talltd t*ttst was applltd to difftrtncts bttwttn avtragt txptrlmtntal and control 
group outcomts. Statistical signlficanct Itvtis art Indlcattd as ■ 1 ptrctnt or 
loss; *^ ■ 5 ptrctnt; * ■ 10 ptrctnt. 
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FIGVRE 4.3 
MONTHLY EARNINGS 
FOR EXPERIMENTALS AND CONTROLS 



Earnings per Month ($) 
400 
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SOURCE: Toble 4.6. 
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TABLE 4.7 
IMPACTS ON PRODUCTIVE ACTIVITY 



OMtCMt 


ExporlntntAls 


Controls 


Difftrtnct 
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Or AXXOIIQfQ A COI IV^V Or pOlX-lSCOfKISrj 
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icffiooi Quri n9 










liiMA 1 AAA 


Af\ 1 
#U* 1 






fl MCI 


July 


49.7 


52.5 


-2.8 


0.316 


August 


65.2 


64.8 


0.4 


0.880 


StptMbtr 


78.0 


76.7 


1.3 


0.581 


Octobtr 


79.2 


79.0 


0.2 


0.926 


NovMbor 


81.5 


81.0 


0.6 


0.798 


DtCMbtr 


82.9 


81.7 


1.2 


0.S78 


JaiHiary 1M9 


84.2 


84.0 


0.2 


0.939 


Fobruary 


85.3 


84.7 


0.7 


0.738 


Itareh 


86.6 


84.7 


1.8 


0.345 


April 


87.3 


84.5 


2.8 


0.143 


Itay 


85.5 


84.8 


0.7 


0.721 


S«ii9lo SI 10 


621 


612 







SOURCES: NORC calculations from Carttr Btglnnlngs tnrollMnt fom and 1989 
survoy data. 



NOTES: Calculations for this tabit usad solf-rtportad data for all 621 
txparlMntals and all 612 controls Mho rtspondad to both tht 1988 and 1969 survtys. 
Including thoso with valuos of itro for outcoMS and thost who wtrt ass^gnad to 
Carttr itglnnlngs but did not partlclpatt. 

Avtragt txptrlntntal and control group outcoMs rtporttd htrt art 
adjusttd Mans froi a llntar analysis of covarlanct proctdurt controlling for 24 
kinds of difftrtnct In characttrl sties btfort randon asslgnMnt (stt Ostlt. 1975. p. 
461; Cavt» 1987). itcaust of rounding, thtrt My bt slight discrtpanclts In rtporttd 
SUM and difftrtncts of thtst adjusttd Mans. 

*Tht eoluM labtltd "p" Is tht statistical signlflcanct Itvtl of tht 
difftrtnct bttwttn txptrlMntal and control group outcoMs: that Is. p Is tht 
probability that avtragt outcoMs art difftrtnt only btcaust of random trror. A 
two-talltd t-ttst was applltd to difftrtncts bttwttn avtragt txptrlMntal and control 
group outcoMs. Statistical signlflcanct Itvtis art Indlcattd as ■ 1 ptrcent or 
loss; ■ S ptrctnt; * ■ 10 ptrctnt. 
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school graduation, with college attendance assuming increased importance thereafter. It also 
appears that a substantial number of sample members combined schooling with at least part- 
time emplqyroent* 

Vm. Otbtr Pfomm Impacts 

Table 4.8 reports the impacts of Career Beginnings on other outcomes of interest 
Program planners and cvaluators speculated that participation in a program designed to foster 
college attendance and employment would induce young people to delay family formation. In 
fact, control group members were more likely than their experimental group counterparts to 
have married by the time of the follow-up interview, although marriage rates for both groups 
were low (2 percent for experimentals and 3.6 percent for controls, a statistically significant 
difference). There were no statistically significant differences between the groups in the 
proportions reporting having children of their own or being primarily responsible for child care. 

l\venty percent of the experimentals and 15 percent of the controls were living away 
Erom home in a college dormitory in February 1989. This difference was statistically significant 
and reflects in part the fact that experimentals were more likely than controls to be enrolled 
in college during that month. It may also be that experimentals were more likely than controls 
to attend colleges outside their home communities, although the survey did not address this 
issue directly. 

DC Effect! jpf Experimentals Versus Effects for Participants 

Data presented in the last chapter (Table 3.2) showed that research status did not 
correspond in all cases with program participation or nonparticipation. Of the 621 
experimentals in the study sample, 20.5 percent to!d survey interviewers that they had not 
participated in the program. Of the 612 controls, 69 or 113 percent, reported they had 
participated in the program, although 9 of those 69 then answered "zero" to a subsequent 



^r example, in November 1988, 81.5 percent of the experimentals were engaged in a productive 
activity (defined as being employed or in college or vocational training or in the miliury). Table 4.2 
indicates that about one-half attended college or vocational training during November (a figure that may 
be slightly too high, since it double-counts a handful of students who may have been in both college and 
vocational sdiool during the month), while according to Table 4.5 about one-half were employed during 
the month. Since only 82 percent of experimentals were engaged in 'productive activity" in that month, 
many respondents were both in school and employed during the month. 




TABLE 4.a 
IMPACTS ON OTHER OUTCOMES 



Exptrl Mnt«1 s Control s 
OutcoiH (%) (%) Dlfftrtnct p* 

In Itay 1989 

Ntrrltd 2*0 3.6 -1.7* 0.077 

H«s own chlld(rtn) 13.7 15.4 -1.6 0.363 

PrlMrlly rtsponslbit for child care 10.1 11.6 -l.S 0.363 

In Ftbruary 1969, living In 
HOM of partnt or guardian 67.7 68.2 *0.6 0.830 
Domltory 20^3 15^3 5.0** 0.014 

Sawpit Sl2t 621 612 

SOURCES: NORC calculations from Carttr Beginnings enrol Imant form and 1989 
survey data. 

NOTES: Calculations for this table used self-reported data for all 621 
experl mentals and all 612 controls Mho responded to both the 1988 and 1989 surveys. 
Including those with values of zero for outcomes and those who were assigned to 
Career Beginnings but did not participate. 

Average experimental and control group outcomes reported here are 
adjusted means from linear analysis of covarlance procedure controlling for 24 kinds 
of difference In characteristics before random assignment (see Ostle, 1975, p. 461; 
Cave. 1987). Because of rounding, there may be slight discrepancies In reported sums 
and differences of these adjusted means. 

*The column labeled "p" Is the statistical significance level of the 
difference between experimental and control group outcomes: that Is. p Is the 
probability that average outcomes are different only because of random error. A 
two-tailed t-test was applied to differences between average experimental and control 
group outcomes. Statistical significance levels are Indicated as *** • 1 percent or 
less; • 5 percent; * • 10 percent. 
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question about the number of specific Ca. eer Beginnings activities in which they had taken 
part Thus, between 9.8 percent and 11.3 percent^ of controls can be assumed to have 
participated in the program. 

In this section, we look at one impact measure, college attendance, to try to understand 
the contributions of these four categories of youths (experimentals who participated, 
experimentab who did not participate, controls who participated, and controls who did not 
participate) on the impacts, and the possible magnitude of any adjustments that might pertain. 

If it is fair to assume that the effect of Career Beginnings on college attendance vm 
zero for those assignees to the program who never participated, then the 4.7 percentage point 
average impaa on college attendance is an average of zero for 20.5 percent of them, and 
4.7/0.795 s 5.9 percentage points for those who did participate. Although subject to some 
question, this adjustment is one that seems empirically correct and is frequently made for 
analyses such as these. 

Adjustments for controls who participated in Career Beginnings are more suspect, sinuc, 
for example, the youths may not have distinguished between the Career Beginnings applicavion 
process and the receipt of Career Beginnings program services. Thus the most optimistic 
approach wov^d be to assume further that impacts on participating controls were exactly the 
same as impacts on participating assignees to the program. With that assumption* the 4.7 
percentage point average impact of assignment comprises of the difference between the 
experimental group's participation rate of 0.795 and a participation rate range of 0.098 to 
0.113 for the controls. An upper-bound estimate of the impact of participation, given this 
further assumption, is 6.9 percentage points. The estimate of 6.9 percentage points requires 
assumptions about both nonparticipants and served controls that are the most favorable to the 
Career Beginnings program. The assumption on control services is particularly liberal in view 
of the high proportion of served controls who r'^ceived only one or two Career Beginnings 
services. While only about one-quarter (129 of 494) of the experimentals who reported 
receiving one or more services received only one or two, two-thirds (40 of 60) of the controls 
jvho reported receiving one or more services got only one or two. A compromise assumption 



'|r69 - 9)/612 m 9.8 percent; 69/612 > 11.3 percent. 

^ Vor cnrnple, if a site 'channeled' the selected youth into other programs and services as well, there 
oouM have been an effea just from being assigned to Career Beginnings, even without participating in 
Career Beginnings per se. 

"4.7/(0.795 - 0.113) - 6.9. 
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micht be that imptcts were the same as impacts for participating experimentals only for those 

20 controls Mtbo took part in three or more Career Beginnings services. On that basis, the 

adjusted impact of participating in the program becomes 62 percenuge points^^. 

As suggested above, these adjusted estimates are only as good as the assumptions 

underlying them. If, for example, the effect of the program on participating controls was even 

weaker than under the second assumption (because they tended to receive fewer services), or 

if those selected for the program tended to think more highly of themselves as 'college 

material," even without attending any activities, the true impact of participation would be even 

closer to the impact for all those assigned to the experimental group. The impact of 

assignment thus is a more conservative estimate of the effect of the program than the impact 

of participation calculated on any of the assumptions just described. 

• • • 

Young people who were given the opportunity to participate in Career Beginnings 
attended college at a higher rate than would otherwise have been the case. They also 
appeared somewhat more likely to opt for higher levels of education, with some of those who 
would have attended vocational schools instead going to junior colleges and some of those who 
woukl have attended two-year colleges instead selecting four-year institutions. 

That the increase in college attendance was not larger than 4.7 percentage points may 
reflect the fact that, as discussed in the previous chapter, individuals in the control group also 
received substantial amounts of service. This 4.7 percentage point overall increase is an 
average of site differences that ranged from -3.0 percentage points (that is, controls were more 
likely to attend college than oq)erimentals) to 143 percentage points. The sites registering the 
largest impacts were also the most successful in delivering more services to experimentals than 
they would otherwise have received 

Appendix C presents further findings on the association of receipt of services (whether 
from Career Beginnings or other sources) and colLge-going. As it makes plain, it may be 
impoiSi1)le to accurately measure the impaa of the receipt of any services on college-going 
because those participating in services differ from nonparticipants in many unmeasured — and 
unmeasurable - ways. 



^^4.7/(0.795 - 20/60) « 6.2 
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The aqwctatkm that aperimentab' increased investment in education would be associated 
with focfooe employment and eanunp in ihe short run also was borne out by the nxncy data, 
though the dififerences disappeared midway throu^ the foUow-up year. 
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APPENDIX A: SUPPLEMENT TO CHAPTER 2 
GICAL ISSUES IN THE IMPACT ANALYSIS OF CAREER BEGINNINGS 



As outlined in Chapter 2, random assignment is used to avoid systematic differences in 
pre-program characteristics of experimental and control groups. This appendix compares the 
characteristics of experimentals and controls for the entire sample and for individual sites. It 
also compares the characteristics of those who responded to follow-up surveys with those who 
did not respond so that it can be seen whether results for responden can be generalized to 
the entire sample. 

I. Selecttoii Bias 

Did random assignment succeed in creating a group of Career Beginnings controls who 
had the same pre-program characteristics as Career Beginnings experimentals? Table A.1 
presents linear regression results measuring the extent of selection bias for several different 
samples, including all 1^74 persons who filled out enrollment forms as well as the responders 
to both surve>s« who form the impact analysis sample. In this regression, the dependent 
variable was unity for each experimental and zero for each control. Individual characteristics 
and the site at v^ch the person applied for the program were the independent variables. 

The column labeled "Full Sample*" in Table A.1 shows the same slight differences in 
individual characteristics as Table 11 in Chapter 2, which presented one-at-a-time average 
characteristics for experimentab, controls, and both groups together. The Gnal entry in the 
column, the p value of the F statistic, indicates that there is a 45 percent probability that the 
overall measured characteristics were the same for the experimentals and controls in the full 
sample - the 1,574 persons filled out an intake form. 

The impact results presented in this report are based on the 1,233 persons who 
responded to both follow-up surveys. The p value of the F statistic, presented at the bottom 
of the right column of Table A.1, indicates that there is a 73 percent probability that the 
overall measured characteristics were the same for experimentals and controls. 

Tables A2 tlirough A.8 compare experimentals and controls within each site. As would 
be expected with small sample sizes, there were differences in the two groups within a site. 

^7- 



n* NoBrciiioMc Bias 

Were those persons y^ho responded to both the one-year and two-year follow-up surveys 
representative of the full sample of all persons randomly assigned? A high degree of mobility 
among disadvantaged youths makes it diCQcult for survey interviewers to locate all of them a 
year or two after they have become part of a study. Of the 1,574 people in the fiiU sample, 
1,463 responded to the first-year survey, 1»277 responded to the second-year survey, and 1,233 
responded to both surveys. Overall, 783 percep' of the sample responded to both surveys; for 
experimentals the response rate was 77.7 percent, while for controls it was 79.0 percent. 

Table A.9 presents linear regression results measuring the extent to which average 
characteristics at random assignment for survey responders differed from those of 
nonresponders. The middle column in the table is most important, since it compares those 
who responded to both surveys (and were thus part of the impact sample) with those who 
did not Since the final entry in this column, the p v^^Hie of the F statistic, is zero to three 
decimal places, there is strong evidence of systematic differences between those who were in 
the impact sample and those who were not Respondents to both surveys were more likely to 
be female, living with both parents, not parents themselves, and taking mostly college 
preparatory courses. They were also less likely to have been employed during the previous 
school year or to have been expelled or suspended. The response rate at Jacksonville was also 
significantly lower than at other sites, even after taking into account differences in individual 
characteristics. However, the proportion of experimentals was not significantly different 
between those who responded to both surveys and nonresponders to one or both surveys. 

These findings imply that the program impact results for responders to both surveys 
cannot be easily generalized to the full sample. However, since responders to both surveys 
constituted such a high percentage of all Career Beginnings youths, the findings arc 
representative of a broad group of the sample. 




TABLE A.l 



ESTIMATED REGRESSION COEFFICIENTS FOR PROBABILITY OF 
ASSIGNMENT TO EXPERIMENTAL GROUP 



Sample 



Regressor or Statistic 



Full Sample 



1989 Survey 
Responders 



1988 Survey 
Responders 



Responders to 
Both Surveys 



Constant 0.508** 

Male -0-041 
Site 

The Bronx, New York 0.030 

Gary. Indiana 0.027 

Indianapolis. Indiana 

Jacksonville. Florida 0.075 

Rochester, New York 0.018 

Santa Ana, California 0.024 

Youngstown. Ohio 0.037 

Employed anytime during the 

year before random assignment 0.034 

Recent grade average 

C, D, or F 0.009 

Ever a school dropout 

for a semester or more -0.032 

Problems with English and 

another language Is usually 

spoken at home -0.015 

Living with both parents 0.014 

Has own chlld(ren) -0.082 

Ever in a simmer work 

program -0.018 

Employed anytime during 

the previous school year 0.017 

Ever expelled j ' suspended -0*003 



0.502*** 
-0.049 



-0.014 
0.004 

0.029 
-0.026 
0.035 
0.002 



0.040 
0.011 
-0.025 

-0.003 
0.009 
-0.113 

-0.021 

0.005 
0.006 



0.507*** 
-0.032 



0.022 
0.014 

0.069 
0.008 
0.017 
0.021 



0.022 
0.009 
0.045 

-0.032 
0.020 
-0.061 

-0.022 

0.012 
0.015 



0.504*** 
.0.045 



-0.017 
0.002 

0.033 
-0.040 

0.030 
-0.000 



0.033 
0.007 
0.008 

-0.013 
0.013 
-0.094 

-0.021 

0.005 
0.019 



(continued) 



TABLE A.l (cont1nu«d) 







1989 Survey 


1988 Survey 


Responders to 


Regrfssor or Statistic 


Full Sample 


Res ponders 


Responders 


Both Surveys 


tinnicixy 










Whitt, non-Hlspanlc 


•0.086* 


-0.li37 


-0.098* 


-0.097* 


Hispanic 


•0.065 


-0.046 


-0.067 


-0.058 


DiacK or oxncr 










Family racalving cash welfare 


0.021 


0.007 


0.019 


0.007 


Aga 18 or older 


•0.012 


-0.021 


-0.024 


-0.030 


UOMrsva wmf9Vtj wuiiey^c 










preparatory 


•0.030 


-0.044 


-0.012 


-0.039 


















-0 072** 


-0 041 


rwHi ijf rcfcvivin^ rowu dbflinpa 






•W. Wfcl 




fcwVI III V *iAHI>HI Wi VI III 11^ 










piwyiVN fflWW •|AMI«UI W 11^ kllV 










• VNi|l 1 V nMMDVi a iilyii swInlOl 






W. A AO 


-0 088* 
"w . www 


NiMiber of observations 


1574 


1277 


1463 


1233 


Number of experimental s 


799 


641 


742 


621 


Number of controls 


775 


636 


721 


612 


Degrees of freedom for error 


1549 


1252 


1438 


1208 


Error mean square 


0.250 


0.251 


0.250 


0.251 


R square 


0.015 


0.015 


0.016 


0.016 


Nean of dependent variable 


0.508 


0.502 


0.507 


0.504 


F statistic 


1.008 


0.799 


0.974 


0.808 


P value of F statistic 


0.452 


0.741 


0.498 


0.729 



S(MJRCE: MORC calculations from Career Beginnings enrollment form data. 

NOTES: The dependent variable In each regression equation was unity for each experimental and 
zero for each control. Each characteristic on the right-hand side of each equation was measured as a 
deviation from Its mean. 

A two-tailed t-test was applied to each coefficient estimate. Statistical 
significance levels are Indicated as « 1 percent or less; « 5 percent; * « 10 percent. 

The p value of the F statistic Is the probability of obtaining these coefficient 
estimates If the true chance of becoming an experimental did not vary with any characteristic. Thus, 
the closer the p value Is to unity, the more successful was random assignment In equating average 
characteristics of experimental s and controls. 



TABLE A. 2 

CHARACTERISTICS AT SAMPLE ENTRY, BY RESEARCH GROUP: THE BRONX, HEW YORK 



Both 

Sample Experimental s Controls Groups 
Characteristic and Subgroup Siie (%) (%) W p' 



Sax 



Female 


118 


73.2 


65.9 


69.4 


Hale 


S2 


26.8 


34.1 


30.6 












th« ytar btfort random 










isslgfMtnt 








41.2 


Yts 


70 


39.0 


43.2 


No 


100 


61.0 


56.8 


58.8 


Rectnt grade average 










A or A- 


18 


14.8 


6.9 


10.7* 


B* 


ss 


29.6 


35.D 


3Z./ 


B or B- 


S3 


27.2 


35.6 


31.5 




31 


24.7 


12.6 


18.5 


C or C- 


s 


3.7 


2.3 


3.0 


D+ 


1 


0.0 


1.1 


0.6 


D or D- 


2 


0.0 


2.3 


1.2 


F 


3 


0.0 


3.4 


1.8 


Ever a school dropout 










for a senester or more 










Yes 


4 


1.2 


3.4 


2.4 


No 


166 


98.8 


96.6 


97.6 


Problem with English and 










another language Is usually 










spoken at home 








0.6 


Y#s 


1 


0.0 


1.1 


No 


169 


100.0 


98.9 


99.4 


Living with both parents 








31.8 


Yes 


S4 


30.5 


33.0 


No 


116 


69.5 


67.0 


68.2 


Has own child(ren) 








0.6 


Yes 


1 


0.0 


1.1 


No 


169 


100.0 


98.9 


99.4 


Ever in a summer work 










prograa 








28.8 


Yes 


49 


29.3 


28.4 


No 


121 


70.7 


71.6 


71.2 



0.390 



0.693 



0.071 



0.664 



1.000 



0.857 



1.000 



1.000 



(continued) 



TABLE A.2 (contlnutd) 



Both 





Stmplt 


Experimental s 


Controls 


Groups 




Chiracttrlstic «nd Subgroup 


Size 


(*) 


(*) 


(%) 


p' 














tht prtvlous school yt«r 












Yts 


35 


19.5 


21.6 


20.6 


0.885 


No 


135 


60.5 


78.4 


79.4 




Evtr txpolltd or suspondtd 












Yts 


7 


2.4 


5.7 


4.1 


0.498 


No 


163 


97.6 


94.3 


95.9 




Ethnicity 












Nhlttt non-Hispanic 


6 


4.9 


2.3 


J. 5 


0.215 


Black, non-Hispanic 


93 


58.5 


51.1 


54.7 




Hispanic 


67 


32 9 


45 5 


A 

Ja> . ^ 




AmH can Indian 


1 


0.0 


1.1 


0.6 




Asian or Pacific Islander 


2 


2.4 


0.0 


1.2 




Othtr 


1 


1.2 


0.0 


0.6 




FaMlly rtcelving cash welfare 












Yes 


46 


29.3 


25.0 


27.1 


0.650 


No 


124 


70.7 


75.0 


72.9 




Age 18 or older 












Yts 


13 


3.4 


12.2 


7.6* 


0.062 


No 


157 


96.6 


87 8 


Q? a 




Coursts 












Gtntral 


74 


42.3 


50.6 


46.5 


0.299 


Colltgt prtparatory 


24 


11.5 


18.5 


15.1 




CoHMrclal or business 


36 


25.6 


19.8 


22 6 




Vocational or ttchnlcal 


23 


19.2 


9.9 


14.5 




Othtr 


2 


1.3 


1.2 


1.3 




Partnt(s) graduated from or 












attended college 












Yes 


60 


36.6 


34.1 


35.3 


0.858 


No 


110 


63.4 


65.9 


64.7 




Fanlly receiving Food Stamps 












Vac 

Tes 




£9.9 


Z3.9 


24.7 


0.932 


No 


128 


74.4 


76.1 


75.3 




Evtr In « yttr-round training 












program not sponsorod by the 












sMpIt MMbcr's high school 












Yts 


13 


7.3 


8.0 


7.6 


1.000 


No 


157 


92.7 


92.0 


92.4 




SMpIt Size 


170 


82 


88 







(continued) 
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TABLE A.2 (continued) 



SOURCE: MDRC calculations from Career Beginnings enrollment form data. 

NOTES: Calculations for this table used self -reported data for all sample members in 
the site who responded to both the 1988 and 1989 surveys. Subgroup sample sizes for some 
characteristics may fall short of the total number of scimple members because of Items 
missing from data for some survey responders. 

Because of rounding, distributions may not total exactly 100.0 percent. 

*The column labeled "p** Is the statistical significance level of the 
difference between the distributions of characteristics for the experimental and control 
groups: that is, p is the probability that distributions are different only because of 
random error. A dash Indicates that no p value was calculated since all persons had the 
same response to this question. A Pearson chl-square test was used to test the hypothesis 
of equal distributions. Statistical significance levels are indicated as > 1 percent or 
less; » 5 percent; ♦ » 10 percent. 
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TABU A. 3 

CHARACTERISTICS AT SAMPLE ENTRY. BY RESEARCH GROUP: GARY. INDIANA 



Both 



Chtracttrlstic and Subgroup 


Sanplt 
Sizt 


Exptrlatntals 

(*) 


Controls 
(*) 


Groups 
(*) 


P« 


Stx 

P«Hlt 


111 
68 


65.6 
34.4 


58.4 
41.6 


62.0 
38.0 


0.407 


Etapleytd anytlM during 
tht ytar btfort rtndoiR 
migntnt 

Ym 

No 


64 

115 


34.4 

65.6 


37.1 
62.9 


35.8 
64.2 


0.832 


Rtctnt grtdt average 
A or A- 
B* 

B or B- 

C* 

C or C- 

D or D- 
F 


8 
19 
52 
44 
50 
4 
2 
0 


3.3 
5.6 
33.3 
24.4 
26.7 
4.4 
2.2 
0.0 


5.6 
15.7 
24.7 
24.7 
29.2 
0.0 
0.0 
0.0 


4.5* 
10.6 
29.1 
24.6 
27.9 
2.2 
1.1 
0.0 


0.061 


Ever a school dropout 
for a stntsttr or more 
Yts 

No 


2 
177 


0.0 
100.0 


2.2 
97.8 


1.1 
98.9 


0.472 


Probltm with English and 
another languagt Is usually 
spoktn at hoM 

Yts 

No 


0 
179 


0.0 
100.0 


0.0 
100.0 


0.0 
100.0 




Living with both parents 
Yts 
No 


74 

105 


42.2 
57.8 


40.4 

59.6 


41.3 
58.7 


0.929 


Has own ch11d(rtn* 
Yts 
No 


3 
176 


1.1 
98.9 


2.2 
97.8 


1.7 
98.3 


0.992 


Evtr In a SMMtr work 
progrtB 

Yts 

No 


90 
89 


48.9 

51.1 


51.7 
48.3 


50.3 
49.7 


0.822 



O . -94 



ERIC 
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TABLE A. 3 (contlnutd) 



Both 



Ch«r«cttr1st1c and Subgroup 


Sample 
Sizt 


ExperliNntals 
(%) 


Controls 
(%) 


Groups 
(%) 




Eiiploytd anytime during 
vfiv prwiwus scnool jrvor 

Yts 

No 


2S 
1S4 


11.1 
88.9 


16.9 
83.1 


14.0 
86.0 


0.372 


Evtr txpclltd or suspended 
Yts 

No 


26 

ISl 


16.7 
83.3 


14.3 
85.7 


15.6 
84.4 


0.862 


Whittt non-Hispanic 
B1ack» non-Hispanic 
Hispanic 

Mwvriwvn I flu ion 

Asian or Pacific Islander 
Other 


3 

172 
» 

1 

0 
0 


3.3 
93.3 
3.3 
0 0 
0.0 
0.0 


0.0 
98.9 
0.0 
1.1 
0.0 
0.0 


1.7* 
96.1 
1.7 
0.6 
0.0 
0.0 


0.069 


romi ijr rvcvivin^ cosn wciTorc 

Yes 

No 


66 
113 


40.0 
60.0 


33.7 
66.3 


36.9 
63.1 


0.473 


Age 16 or older 
Yes 
No 


19 
160 


11.1 
88.9 


10.1 
89.9 


10.6 
89.4 


1.000 


Courses 
General 

College preparatory 
Connerclal or business 
fowavioiiai or vccnniwoi 
Other 


39 
66 

23 

3 


29.5 
44.3 
13.6 

10.2 
2.3 


15.1 
54.7 
12.8 
16.3 
1.2 


22.4 
49.4 

13.2 
13.2 
1.7 


0.164 


Parent (s) graduated from or 

W WiNi VM WW 1 1 

Yts 
No 


77 
102 


40.0 
60.0 


46.1 
53.9 


43.0 
57.0 


0.504 


Family recti ving Food Stamps 
Yts 
No 


6S 
114 


37.8 
62.2 


34.8 

09.2 


36.3 
63.7 


0.799 


in m — 1 vuini viviniii^ 

progrM not sponsored by the 
sample nenber's high school 
Yes 

No 


6 

173 


3.3 
96.7 


3.4 
96.6 


3.4 
96.6 


1.000 


Stniple Size 


179 


90 


89 







SOURCE AND NOTES: See Table A. 2. 
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TABLE A.4 

CHARACTERISTICS AT SAHPLE ENTRY. BY RESEARCH GROUP: INDIANAPOLIS. INDIANA 



Both 

SMpIt ExperlMtntals Controls Groups 
CNracttrlstic and Subgroup Sizt (%) (%) (%) P* 



Stx 



F«Mlt 

Nal* 


144 
68 


66.1 
33.9 


70.0 
30.0 


67.9 
32.1 


0.642 


Enploytd anytime during 
the yMr before random 
assignment 

Yes 

No 


185 
27 


87.5 
12.5 


87.0 
13.0 


87.3 
12.7 


1.000 


Recent grade average 
A or A- 
B^ 

B or B- 
C* 

C or C- 
D* 

0 or D- 
F 


6 

18 
47 
48 
78 
8 
7 
0 


3.6 
8.0 
25.0 
24.1 
34.8 
1.8 
2.7 
0.0 


2.0 
9.0 
19.0 
21.0 
39.0 
6.0 
4.0 
0.0 


2.8 
8.5 
22.2 
22.6 
36.8 
3.8 
3.3 
0.0 


C.594 


Ever a school dropout 
for a semester or more 

Yes 

No 


6 

206 


3.6 
96.4 


2.0 
98.0 


2.8 
97.2 


0.784 


Problems with English and 
another language Is usually 
spoken at home 

Yes 

No 


0 
212 


0.0 
100.0 


0.0 
100.0 


0.0 
100.0 




Living with both parents 
Yes 
No 


S6 

156 


28.6 
71.4 


24.0 
76.0 


26.4 
73.6 


0.550 


Has own chlld(ren) 
Yes 
No 


9 

203 


3.6 
96.4 


5.0 
95.0 


4.2 
95.8 


0.862 


Ever In a sunmer work 
program 

Yes 

No 


199 
13 


92.0 
3.0 


96.0 
4.0 


93.9 
6.1 


0.349 



(continued) 
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TABLE A.4 (centlnutd) 



Both 



Ch«r«cttr1st1c and Subgroup 


Sample 
Size 


ExDerl mental s 

(*) 


Control s 
(%) 


Groups 
(%) 


P» 


Biiploytd anytlM during 

Yts 

No 


80 

132 


39.3 
60.7 


36.0 
64.0 


37.7 
62.3 


0.726 


Evtr txptlltd or suspended 
Yes 
No 


67 
145 


28.6 
71.4 


3S.0 
65.0 


31.6 
68.4 


0.391 


Mi1tt» non-Hi spanic 
Black. non-Hispanic 
Hispanic 

AiMrl tnHtiin 

Asian or Pacific Islander 
Other 


6 

203 
2 

A 
V 

1 

0 


3.6 
95.5 
0.0 
0.0 

o.s 

0.0 


2.0 
96.0 
2.0 
0.0 
0.0 
0.0 


2.8 
95.8 
0.9 
0.0 
0.5 
0.0 


0.309 


Yas 

No 


72 
140 


32.1 
67.9 


36.0 
64.0 


34.0 
66.0 


0.655 


Aga 18 or oldar 
Yas 
No 


10 
202 


1.8 
98.2 


8.0 
92.0 


4.7* 
95.3 


0.071 


Coursas 
Ganaral 

Coll aga praparatory 
Conaarclal or buslnass 

VA^Aflonftl or tA^hnlfAl 

Other 


101 
56 
20 
27 
2 


55.0 
23.9 

7.3 
12.8 

0.9 


42.3 
30.9 
12.4 
13.4 
1.0 


49.0 
27.2 

9.7 
13.1 

1.0 


0.404 

• 


Parent (s) graduated from or 

AftAllHMi Cnl 1 

Yas 

No 


47 

165 


20.5 
79.5 


24.0 
76.0 


22.2 
77.8 


0.660 


Family racalving Food Stamps 
Yas 
No 


72 
140 


33.0 
67.0 


35.0 
65.0 


34.0 
an A 


0.876 


Pir*r In A VAAP*roiinH trAlnIno 

kWI III m yWml 1 VUllli W 1 Villi iiy 

progren not sponsored by the 
sanple Mnber's high school 

Yes 

No 


14 

196 


7.1 
92.9 


6.0 
94.0 


6.6 
93.4 


0.9S4 


Sample Size 


212 


112 


100 







SOURCE AND NOTES: See Table A.2. 
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TABLE A. 5 

CHARACTERISTICS AT SAHPLE ENTRY. BY RESEARCH GROUP: JACKSONVILLE. FLORIDA 



Both 



Ch«r«cttr1st1c and Subgroup 


Sample 
Size 


Experimental s 

(%) 


Controls 
(%) 


Groups 
(%) 




Stx 

F«Mlt 

Malt 


109 
44 


71.4 
28.6 


71.0 
29.0 


71.2 
28.8 


1.000 


Eiiploytd mytlM during 
th« y#«r btfort random 
assignntnt 

Yts 

No 


SI 

102 


39.3 
60.7 


26.1 
73.9 


33.3 
66.7 


0.121 


Rtctnt grtde average 
A or A- 

B or B- 

C or C- 
0+ 

0 or 0- 
F 


9 
16 
4S 
36 
42 
2 
3 
0 


7.1 
7.1 
26.2 
21.4 
33.3 
2.4 
2.4 
0.0 


4.3 
14.5 

33.3 
26.1 
20.3 
0.0 
1.4 
0.0 


5.9 
10.5 
29.4 
23.5 
27.5 
1.3 
2.0 
0.0 


0.267 


Evtr a school dropout 
for a samtsttr or more 

Yts 

No 


1 

152 


1.2 
98.8 


0.0 
100.0 


0.7 
99.3 


1.000 


Probltns with English and 
anothtr Itnguagt Is usually 
spoktn at hont 

Yts 

No 


1 

152 


1.2 
98.8 


0.0 
100.0 


0.7 
99.3 


l.OC/0 


Living Mith both parents 
Yts 

Mo 


S7 
96 


36 1 
61.9 


36.2 
63.8 


37.3 
62.7 


0.945 


Hat OMn chlld(ran) 
Yts 
No 


5 

148 


4.8 

95.2 


1.4 
98.6 


3.3 
96.7 


0.490 


Evtr In a sunnier work 
prograa 

Yts 

No 


48 

105 


31.0 
69.0 


31.9 
68.1 


31.4 
68.6 


1.000 



(continued) 
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TABLE A. 5 (continued) 



Both 

Sanplt Exptrlnenttls Controls Groups 



Ch«r«cttr1st1c and Subgroup 


Sizt 


(%) 




(%) 


P' 


Employtd anytiM during 
tht prtvlous school ytar 

Yts 

No 


40 

113 


26.2 
73.8 


26.1 
73.9 


26.1 
73.9 


1.000 


Evtr txpolltd or suspended 
Yes 
No 


29 

.24 


25.0 
75.0 


11.6 
88.4 


19.0* 
81.0 


0.058 


Ethnicity 
tlh1te» non-Hispanic 
Black. non-Hispanic 
Hispanic 
American Indian 
Asian or Pacific Islander 
Other 


12 
140 

0 
0 

1 
0 


3.6 
96.4 
0.0 
0.0 
0.0 
0.0 


13.0 
85.5 
0.0 
0.0 
1.4 
0.0 


7.8** 
91.5 
0.0 
0.0 
0.7 
0.0 


0.049 


Family receiving cash welfare 
Yes 
No 


22 
131 


16.7 
83.3 


11.6 
88.4 


14.4 
85.6 


0.510 


Age 18 or older 
Yes 
No 


9 
144 


2.4 
97.6 


10.1 
89.9 


5.9* 
94.1 


0.092 


Courses 
General 

College preparatory 
CoMercial or business 
Vocational or technical 
Other 


83 
S2 
3 
9 
3 


63.4 
28.0 
2.4 
4.9 
1.2 


45.6 
42.6 
1.5 
7.4 
2.9 


55.3 
34.7 
2.0 
6.0 
2.0 


0.238 


Parent (s) graduated from or 
attended college 

Yes 

No 


63 
90 


34.5 
65.5 


49.3 
50.7 


41.2* 
58.8 


0.093 


Family receiving Food Stamps 

Vac 

No 


140 


w. i9 

91.7 


8 7 
91.3 


8.5 
91.5 


1.000 


Evtr In a ytar-round training 
program not sponsortd by the 
samplt nmbar's high school 

Yts 

No 


9 
144 


4.8 

95.2 


7.2 
92.8 


5.9 
94.1 


0.761 


Samplt Sizt 


153 


84 


69 







SOURCE AND NOTES: Set Table A. 2. 
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TABLE A. 6 

CHARACTERISTICS AT SANPLE ENTRY. BY RESEARCH GROUP: ROCHESTER. NEW YORK 



Chtrtcttrlstic and Subgroup 



Sample Experlawnttls Controls 
Sizt (%) (%) 



Both 
Groups 
(%) 



Stx 
F«Mlt 

Nalt 



100 
74 



55.6 
44.4 



59.1 
40.9 



57.5 
42.5 



0.747 



Enploytd tnytlnt during 
tht yMr btfort random 
asslgmiant 

Yts 

No 



100 
74 



58.0 
42.0 



57.0 
43.0 



57. 
42. 



1.000 



Rtctnt grade average 
A or A- 
B* 

B or B- 
C* 

C or C- 
D* 

0 or D- 
F 



17 
34 
SS 
S2 
9 
6 
0 



1.2 
7.4 
17.3 
35.8 
28.4 
6.2 
3.7 
0.0 



0.0 
11.8 
21.5 
28.0 
31.2 
4.3 
3.2 
0.0 



0.6 
9.8 
19.5 
31.6 
29.9 
5.2 
3.4 
0.0 



0.719 



Ever a school dropout 
for a SMTster or more 

Yts 

No 



0 
174 



0.0 
100.0 



0.0 
100.0 



0.0 
100.0 



Problea^ with English and 
another language Is usually 
spoken at hotae 

Yts 

No 



3 

171 



2.5 
97.5 



1.1 
98.9 



1.7 
98.3 



0.904 



Living with both parents 
Yts 
No 

Has own chlld(ren) 
Yts 
No 



85 
89 



3 

171 



51.9 
48.1 



0.0 
100.0 



4f.2 
53.8 



3.2 
96.8 



48.9 
51.1 



1.7 
98.3 



0.557 



0.295 



Evtr In a sunmer work 
program 

Yts 

No 



54 

120 



34.6 
65.4 



28.0 31.0 0.438 

72.0 69.0 



(continued) 
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TABLE A.6 (contlnutd) 



Both 

Sampl e Experimental s Control s Groups 



Cherect eristic end Subgroup 


Sizt 


(%) 


(%) 


(%) 




GnlanMl AnvtiM durlnO 

the previous school yeer 
Yes 
No 


81 
93 


43.2 
56.8 


49.5 

50.5 


46.6 
53.4 


O.SOl 


Ever expelled or suspended 
Yes 

No 


38 
136 


23.5 
76.5 


20.4 
79.6 


21.8 
78.2 


0.766 


Ethnicity 
White. non-Hispanic 
Black. non-Hispanic 
Hispanic 
American Indian 
Asian or Pacific Islander 
Other 


29 
117 
24 

0 
4 

0 


13.6 
67.9 
16.0 
0.0 
2.S 
0.0 


19.4 
66.7 
11.8 
0.0 
2.2 
0.0 


16.7 
67.2 
13.8 
0.0 
2.3 
0.0 


0.693 


Family receiving cash welfare 
Yes 
No 


47 
127 


29.6 
70.4 


24.7 
75.3 


27.0 
73.0 


0.579 


Age 18 or older 
Yes 
No 


27 
147 


18.5 
81.5 


12.9 
87.1 


15.5 
84.5 


0.418 


Courses 
General 

College preparatory 
Conerclal or business 
Vocational or technical 
Other 


84 
41 

16 
22 
6 


50.6 
19.S 

7.8 
14.3 

7.8 


48.9 
28.3 
10.9 
12.0 
0.0 


49.7* 
24.3 

9.5 
13.0 

3.6 


0.058 


Parent (s) graduated from or 
attended college 

Yes 

No 


i21 


21.0 
79.0 


38.7 
61.3 


30.5** 
69.5 


0.018 


Family receiving Food Stamps 

TtS 

No 


3Z 
142 


Z3.9 

76.5 


86.0 


81.6 




Evtr In a year-round training 
progran not sponsored by the 
saiapit ntmber's high school 

Yts 

No 


20 
154 


8.6 
91.4 


14.0 
86.0 


11. S 
88.5 


0.388 


Sanpit Sizt 


174 


81 


93 







SOURCE AND NOTES: See Table A. 2. 
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TABLE A. 7 

CHARACTERISTICS AT SAMPLE ENTRY. BY RESEARCH GROUP: SANTA ANA. CALIFORNIA 



Both 



Chtrtcttrlstlc and Subgroup 


Sampit 

Sizt 


Exptrlmtntals 

(%) 


Controls 
(%) 


Groups 
(%) 


P" 


Stx 

F«Mlt 


101 
61 


63.1 
36.9 


61.5 
38.5 


62.3 
37.7 


0.966 


Employed anytime during 
tht yMr b«fort random 
assignment 

Yas 

No 


72 
90 


44.0 
56.0 


44.9 

55.1 


44.4 

55.6 


1.000 


Racant arada avaraaa 
A or A- 
B^ 

B or B- 
C* 

C or C* 
0* 

D or D- 
F 


18 

28 

55 
20 

4 
2 
1 


10.7 
16.7 
33.3 
13.1 
20.2 
4.8 
1.2 
0.0 


11.7 
18.2 
35.1 
11.7 
20.8 
0.0 
1.3 
1.3 


11.2 
17.4 
34.2 
12.4 
20.5 
2.5 
1.2 
0.6 


0.666 


Evtr • school dropout 
for a sMwster or more 

Yts 

No 


0 

162 


0.0 
100.0 


0.0 
100.0 


0.0 
100.0 




Problans with English and 
anothtr language Is usually 
spoken at home 

Yes 

No 


41 
121 


23.8 
76.2 


26.9 
73.1 


25.3 
74.7 


0.784 


living with both parents 
Yts 
No 


108 
54 


65.5 
34.5 


67.9 
32.1 


66.7 
33.3 


0.868 


Has own chlld(ren) 
Yts 
No 


1 

161 


0.0 
100.0 


1.3 
98.7 


0.6 
99.4 


0.970 


Evtr In a sumntr work 
program 

Yts 

No 


40 

122 


23.8 
76.2 


25.6 
74.4 


24.7 
75.3 


0.930 



(continued) 
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TABLE A.7 (contlnutd) 



Both 



Ch«r«cttr1st1c «nd Subgroup 


Sample 
Size 


Experimental s Controls 
(%) (%) 


Groups 

(%) 




Ehiploytd •nytlnt during 

YtS 

No 


S3 
109 


36.9 
63.1 


28.2 
71.8 


32.7 
67.3 


0.312 


Evtr txptlltd or suspended 
Yes 
No 


4 

1S8 


3.6 
06.4 


1.3 
98.7 


2.5 
97.5 


0.666 


Ethnlcltv 
Whitt, non-Hispanic 
Blacki non-Hispanic 
Hispanic 

fVRSriwwi inu 1 on 

Asian or Pacific Islander 
Othtr 


3 
9 

74 

A 
V 

76 
0 


2.4 

6.0 
45.2 

0.0 
46.4 

0.0 


1.3 

5.1 
46.2 

0.0 
47.4 

0.0 


1.9 

5.6 
45.7 

0.0 
46.9 

0.0 


0.954 


ram ijT rcwciviiiy wavii tfciiorv 

YtS 

No 


56 
106 


29.8 
70.2 


39.7 
60.3 


34.6 
65.4 


0.242 


Age 18 or older 
YtS 
No 


34 

128 


17.9 
62.1 


24.4 
75.6 


21.0 
79.0 


0.411 


Coursts 
Gtntral 

Colltgt preparatory 
Coamtrcial or business 
VorAtlMAl or technical 
Other 


99 
61 
0 
J 

1 


59.5 
38.1 
0.0 
1.2 
1.2 


62.8 
37.2 
0.0 
0.0 
0.0 


61.1 
37.7 
0.0 
0.6 
0.6 


0.585 


Parent (s) graduated from or 
AttAndftd foil ana 

YtS 

No 


32 
130 


20.2 
79.8 


19.2 
80.8 


19.8 
80.2 


1.000 


Family receiving Food Stamps 
Yes 
No 


43 
119 


21.4 
78.6 


32.1 
67.9 


26.5 
73.5 


0.176 


Ever In e yeer-round trelning 
progran not sponsored by the 
saiiple MMber's high school 

Yes 

No 


19 
143 


9.5 
90.5 


14.1 
85.9 


11.7 
88.3 


0.509 


Sample Size 


162 


84 


78 






SOURCE AND NOTES: See Table A.2. 
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TABLE A.8 

CHARACTERISTICS AT SAMPLE ENTRY. BY RESEARCH GROUP: YOUNGSTOWN. OHIO 

Both 



Characttrlttic and Subgroup 


Sample 
Size 


Experimental s 

(%) 


Controls 
(%) 


Groups 
(%) 




Sm 

F«Mlt 


106 
75 


69.3 
30.7 


49.5 

50.5 


59.0*** 
41.0 


0.010 


Enployad anytliM during 
tht jfMr iMfort random 
asslgnaant 

Ym 

No 


104 
79 


61.4 
38.6 


52.6 
47.4 


56.8 
43.2 


0.297 


fttcent grade average 
A or A- 
B* 

B or B> 
C* 

C or C- 
0* 

0 or 0- 
F 


16 
30 
46 
45 
38 
5 
2 
0 


8.0 
16.1 
33.3 
23.0 
18.4 
1.1 
0.0 
0.0 


9.5 
16.8 
17.9 
26.3 
23.2 
4.2 
2.1 
0.0 


8.8 
16.5 
25.3 
24.7 
20.9 
2.7 
1.1 
0.0 


0.205 


Ever a school dropout 
for a saaester or more 

Yes 

No 


1 

182 


1.1 
98.9 


0.0 
100.0 


0.5 
99.5 


0.969 


Problems with English and 
another language Is usually 
spoken at home 

Yes 

No 


0 

183 


0.0 
100.0 


0.0 
100.0 


0.0 
100.0 




Living with both parents 
Yes 
No 


94 

89 


51.1 
48.9 


51.6 
48.4 


51.4 
48.6 


1.000 


Has own chlld(ren) 
Yes 
No 


6 

177 


3.4 
96.6 


3.2 
96.8 


3.3 
96.7 


1.000 


Ever In a sumner work 
program 

YtS 

No 


15 
168 


6.8 
93.2 


9.5 
90.5 


8.2 
91.8 


0.701 
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TABLE A.8 (contlnutd) 



Both 



Ch«r«cttHst1c «nd Subgroup 


Sanpit 
S^zt 


Exparlnentals 
(%) 


Controls 
(%) 


Groups 

(%) 




Eiiiploytd anytiM during 

Yts 

No 


70 
113 


43.2 
56.8 


33.7 
66.3 


38.3 
61.7 


0.243 


Evtr oxptUtd or susptndtd 
Yts 

No 


54 

129 


30.7 
69.3 


28.4 
71.6 


29.5 
70.5 


0.863 


Fthnleltw 
Hhitt, non*H1sp«n1c 
B1«ck» non*H1spanic 
Hisptnic 

AB#r 4 t nH i an 

AslOT or Pacific Islander 
Othtr 


SS 
115 
9 
1 

* 

2 
1 


25.0 
67.0 
4.5 
1.1 

* • * 

1.1 
1.1 


34.7 
58.9 
5.3 
0.0 
1.1 
0.0 


30.1 
62.8 
4.9 
0.5 
1.1 
0.5 


0.531 


Yts 

Ho 


74 

. 109 


36.4 
63.6 


44.2 
55.8 


40.4 
59.6 


0.352 


Agt 18 or older 
Yts 
No 


12 
171 


4.5 
95.5 


8.4 

91.6 


6.6 
93.4 


0.448 


Coursts 
Gtntral 

Colltgt prtparatory 
CoMMrclal or buslntss 
VocAtlfMiAl or tftchnl caI 

Othtr 


18 
125 

8 
29 

1 


3.4 

79.5 
4.5 

12.5 
0.0 


16.1 
59.1 

4.3 
19.4 

1.1 


9.9** 
69.1 
4.4 

16.0 
0.6 


0.015 


Partnt(s) graduated from or 

Yts 
No 


73 
110 


46.6 
53.4 


33.7 
66.3 


39.9 
60.1 


0.103 


Ftnlly rtctiving Food Stamps 
Yts 
No 


50 
133 


20.5 
79.5 


33.7 
66.3 


27.3* 
7Z./ 


0.066 


Evtr In t yttr^round training 
progrM not sponsortd by tht 
stnpit aanbtr's high school 

Yts 

No 


19 
164 


6.8 
93.2 


13.7 
86.3 


10.4 
89.6 


0.201 


S«ap1t Sizt 


183 


88 


95 






SOURCE AND NOTES: Stt Table A.2. 
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TABLE A.9 



ESTIHATEO REfiRESSION COEFFICIENTS FOR PROBABILITY OF 
UNIT SURVEY RESPONSE 



SMDit and Dtoendtnt Y«rftb1t 


Rtyrtfftor or StotUtIc 


Full Saiiplo 
1969 Survay 
Rasponsa 


Full San^lt 
1988 Survay 
Rasponsa 


Full Saiipla 1988 Raspondars 
Rasponsa to 1989 Survay 
8oth Survays Rasponsa 


1989 Raspondars 
1988 Survay 
Rasponsa 


Constant 


0.820*** 


0.931*** 


0.789*** 


0 847*** 


0.962*** 


ExpoHatntal 


-0.017 


-0.004 


-0.012 


-0.008 


0.007 


Ntit 


-0.121*** 


-0.018 


-0.118*** 


-0.111*** 


-0.001 


Site 

Th« bronxt New York 
Gary» Indfono 
IndUnopo1is» Indiana 
Jacktonvllltt Florida 
Rocht$ttr» Now York 
Santa Ana» California 
YoungttoiMi» Ohio 


-0.077* 
0.019 

-0.107*** 

-0.037 
-0.055 
-0.056 


0.056** 
0.062** 

0.039 
0.057** 
0.004 
0.018 


-0.039 
0.028 

-0.084** 
-0.021 
-0.072 
-0.046 


-0.094** 
-0.028 

-0.124*** 
-0.074** 
-0.079* 
-0.063 


0.045** 

0.011 

0.026 
0.017 
-0.023 
0.010 


Eaploytd anytlaa during tho 
ytar btfort random asslgiwent 


0.017 


0.030* 


0.025 


-0.000 


0.011 


Rtotnt grada avtragt 

C» 0» or F 


-0.018 


-0.006 


-0.018 


-0.013 


-0.001 


Evar a school dropout 
for a saaastor or Mrt 


-0.134 


-0.060 


-0.145 


-0.110 


-0.025 


Froblam iHth English and 
anothar languaga Is usual l^r 
spokan at hoaa 


0.044 


0.005 


0.004 


-0.004 


-0.040 


Living Hth both partnts 


0.067*** 


0.029** 


0.077*** 


0.055*** 


0.015 


Has own ch.ld(rcn) 


-0.096 


0.022 


-0.111* 


-0.135** 


-0.026 


Evar In a swnar work 
program 


-0.037 


0.007 


-0.025 


-0.033 


0.014 


Eaplo/td anytiM during 
tht pravlous school yaar 


-0.038 


-0.007 


-0.045* 


-0.043* 


-0.012 


Evar axpalltd or susptndad 


•0.034 


-0.039** 


-0.050* 


-0.020 


-0.024* 



(continued) 
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TABLE A.9 (continued) 



SmdIc and (koendftnt Varlabit 




Full Sa^plt 


Full Saaplt 


Full Samplt 1988 Rttpondtrt 


1989 Rttpondtrt 




1969 Survty 


1988 Survty 


Rttpontt to 


1989 Survty 


1968 Survty 


••frtssor or SUtlttIc 


Rttpontt 


Rttpontt 


fto^h Slll^AVS 


Rasimmisa 


Rttpontt 












-0.017 


lliitt» non-Hltpanlc 


0.027 


-0.018 




0.031 




0.059* 


-0.014 


0.048 


0.068** 


-0.012 














Faally rtctlvln^ cash iitlfart 


•0.044* 


-0.013 


-0.035 


-0.025 


0.009 


Agt 18 or oldtr 


0.041 


-0.018 




0 063* 


0.001 














prtparatory 


0.024 


0.002 


0 039* 


0.038* 


0.020* 














attMUlMl coll AIM 


0.012 


0.010 


0 oil 


0 001 


•0.002 


PabI 1 W f^^Af tf i HA FOAlf S^ABBC 


0.022 


.0 nng 

— w . wo 


0.009 


0.015 


•0.014 


Tvop i n a VMp*roiifid trai ni no 

^vVI 1 !• • ^^^I • wHf *f Will Illy 


























0.053 


-0.006 


0.051 


0.059* 


-0.001 


IHMbtr of obstrvatlont 


1574 


1574 


1574 


1463 


1277 


lliAbtr of txp^HMntalt 


799 


799 


799 


742 


641 


thalur of controls 


775 


775 


775 


721 


636 


Otgrttt of froadcM for trror 


1548 


1548 


1548 


1437 


1251 


Error Boan tquart 


0.147 


0.065 


0.164 


0.127 


0.033 


R s^rt 


0.056 


0.023 


0.051 


0.055 


C.024 


Naan of dtptndont vaHabIt 


0.811 


0.929 


0.783 


0.843 


0.966 


F tUtlttIc 


3.645 


1.466 


3.305 


3.364 


1.238 


f valut of F tUtittlc 


0.000 


0.065 


0.000 


0.000 


0.194 



SOURCE: MDRC calculatlont fro« Carttr Bt^lnnlngt tnrollntnt fom data. 

NOTES: Tht dtptndtnt vaHabIt In tach rtgrtttlon tquatlon wat unity for tarly attlgraMnt or turvty conplttion, 
and ztro othtrwitt. Each characttrlttic on tht right-hand tidt of tach tquatlon wat ntaturtd at a dtvlatlon froti itt 
Man. 

A two*ta11td t-tttt Mt applltd to tach cotfflcltnt tttlnatt. Statlttlcal tignlflcanct Itvtit art 
Indlcattd at *^ • 1 ptrctnt or Ittt; ** > 5 ptrctnt; * > 10 ptrctnt. A kty rttult In tvtry rtgrtttlon nat that tht 
cotfflcltnt of txptrlMntal ttatut Mt not tlgnlfl^ 'ntly difftrtnt fron ztro. 

Tht p valut of tht F ttatlttic It t\^^ probability of obtaining thttt cotfflcltnt tttlMttt if tht trut 
chanct of ctapltting tht turvty did not vary with any characttrlstlc. Thut» tht clottr tht p valut It to xtro. tht 
sort Important art difftrtnctt In characttrlttlct bttwttn turvty compltttrt and noncoaipltttrt. 
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TABLE B.l 



INPACTS ON COLLEGE AnENOANCE DURING NAY 1989. 
BY SELECTED BASELINE CHARACTERISTICS 







Atttndod s 2- or 














4-Ytsr Colltgt 














Durlna Nav 1989 






Subgroup 






Samplt 


ExptrlMntsIs 


Controls 


Subgroup 




Inpact 




Ch«r«cttr1tt1c «nd Subgroup 


Sizt 


(%) 


(%) 


Impsct 




01 fftrtnct 


«a 

P 


Stx 












•0.4 


0.948 


F«Mlt 


791 


47.6 


43.2 


4.4 


0J74 






Halt 


442 


48.5 


43.7 


4.8 


0.274 






Sitt 














0.510 


Tht Bronx. Now York 


170 


38.3 


41.2 


•2.9 


0.678 




... 


Gory. IndliM 


179 


59.3 


45.7 


13. 6** 


0.046 




... 


IfidlwMOoIlL Indlani 


212 


33.6 


35.8 


-2.1 


0.732 






Jacksonvlllo. Florida 


153 


54.7 


42.3 


12.4* 


0.097 





... 


Rochosttr. Now York 


174 


46.6 


44.1 


2.4 


0.720 


• • " 


... 


Santa Ana* California 


162 


59.6 


54.5 


5.1 


0.480 






Younostown. Ohio 


183 


47.4 


41.9 


5.5 


0.420 






E^)1oyod anytiM during tht 












3.5 




yoar bofort randon asslgnmtnt 












0.505 


Yos 


646 


49.6 


43.4 


6.2* 


0.085 




III 


No 


587 


46.0 


43.3 


2.7 


0.469 






Roctnt gradt avtragt 












-1.9 


0.717 


A or B 


595 


58.3 


54.7 


3.6 


0.342 


• • • 


... 


C. D. or F 


638 


38.2 


32.8 


5.5 


0.133 






Cv#r a achool drooout 
















for • SMMSttr or sort 












8.8 


0.721 


YM 


14 


33.6 


20.4 


13.2 


0.589 






No 


1.219 


48.1 


43.6 


4.4* 


0.089 




... 


ProblMS with English and 
















•nothtr langutgt Is usually 
















spoken St hoM 












9.5 


0.487 


Ym 


46 


49.0 


35.3 


13.7 


0.308 






No 


1.187 


47.9 


43.7 


4.2 


0.115 






Living tflth both psrtnts 












-5.6 


0.287 


YtS 


528 


47.7 


46.3 


1.3 


0.736 






No 


705 


48.1 


41.2 


6.9** 


0.044 







(contlnutd) 
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TABLE B.l (contlnutd) 







Atttndtd t 2- or 
4-Yttr Colltgt 

Darino Nav 1989 






Subgroup 




Charftcttrlstlc ind Suborouo 


Sanplt 
Sizt 


ExptrlMnttIt 

(%) 


Controls 
(%) 


Subgroup 
Inptct 


p* 


I«ptct 
Dlfftrtnct^ 


p« 


Hat own cblld(ran) 
Yts 
Ito 


28 
1.205 


47.; 

47.9 


35.7 
43.6 


11.9 
4.4* 


0.494 
0.097 


7.6 

mmm 


0.668 

mmm 


Evtr In « tmmr Mork 
progrM 

Yts 

No 


495 
738 


44.5 

50.2 


39.7 
45.8 


4.7 
4.4 


0.247 
0.191 


0.3 


0.950 


biploytd anytlM durlns 
tN prtvlout school yttr 
Ym 


384 
849 


43.8 
49.8 


41.8 
44.1 


2.0 
5.7* 


0.668 
0.070 


-3.7 


0.514 


Evtr txptlltd or tuspcndtd 
Ytt 
No 


227 
1.006 


34.5 
51.0 


32.0 
46.0 


2.5 
5.0* 


0.676 
0.083 


-2.5 


0.712 


Uh1tt» non-Hi sp«n1c 
IUck» non-N1sp«n1c 
Hispanic 
Othtr 


114 
849 

179 
91 


42.8 
47.2 
42.9 
69.4 


40.9 
43.5 
42.5 
46.6 


1.9 
3.8 
0.3 
22.8** 


0.824 
0.229 
0.959 
0.017 


— 


0.246 


FMlly rtctlving cash 
Ml f art 

Yts 

No 

www 


383 
850 


45.6 
49.0 


46.0 
42.1 


-0.4 
6.8** 


0.926 
0.030 


-7.3 


0.198 


Agt 

18 or oldtr 
17 or undtr 


124 
1.109 


44.0 
48.3 


41.6 
43.6 


2.4 
4.8* 


0.767 
0.082 


-2.4 

mmm 


0.787 

mmm 


Courttt Bottly colltgt 
prtptrttory 

Ytt 

No 


449 

784 


61.2 
40.3 


56.3 
36.0 


4.9 
4.3 


0.260 
0.185 


0.5 


0.9Z3 


Ptrtnt(t) grtdutttd froa or 
ttttndtd colltgt 

Ytt 

No 


405 
828 


52.0 
46.0 


44.9 

42.7 


7.1 
3.3 


0.116 
0.304 


3.9 


L 17 



(contlnutd) 
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TABLE B.l (eontlnutd) 







Atttnd«d a 2- or 














4-Ytar Colligt 














Durino Miv 198& 












Sanplt 




WMfl \ 1 V 1 • 


Subgroup 




I"P«et 




Chartcttrlstic end Subgroup 


Sizt 






liipact 




Difftrwict'' 


9* 
















0.9O2 




117 


44.6 


42.5 


9 1 


O.OoS 




mmm 


no 


910 


49.0 


43.6 




Q.Q/d 






Evtr In a ytor-round training 
















progrw not sponsortd by tht 
















somplt Mibtr*s high school 












5.9 


0.539 


Ym 


100 


54.5 


44.5 


10.0 


0.280 






No 


1.133 


47.4 


43.3 


4.1 


0.133 







SOURCES: NDRC calculations from Carter Beginnings enrollment form and 1989 survey data. 



NOTES: Calculations for this table used self-reported data for all 621 experlmentals and all 
612 controls who responded to both the 1988 and 1989 surveys. Including those with values of zero for 
outcomes and those who were assigned to Career Beginnings but did not participate. Subgroup sample 
sizes for some characteristics may fall short of the total number of sample members because of Items 
missing from data for some survey responders. 

Average experimental and control group outcomes reported here are adjusted means from 
two-way analysis of covarlance procedures controlling for up to 23 kinds of difference In 
characteristics, other than the characteristic used to define subgroups, before random assignment. 
The two categories used as factors were research assignment and, one at a time, the baseline 
characteristics Indicated (see Ostle, 1975, p. 454). Because of rounding, there may be slight 
discropancles in reported sums and differences of these adjusted means. 

*Two-ta11ed t-tests were applied to w1 thin-subgroup 1im>acts and also to differences 
between subgroup impacts. For each characteristic with more than two subgroups (site and ethnicity), 
an F test wu applied to the Interaction between that characteristic and experimental or control 
status. The columns labeled "p" are the statistical significance levels of each Impact, each 
difference between Impacts, or each F statistic: that Is, p Is the probability that sample estimates 
are non-zero only because of random error. Statistical significance levels are Indicated as 
■ 1 percent or less; ** ■ 5 percent; * ■ 10 percent. 

^For each characteristic that has only two subgroups, the subgroup Impact difference 
Is the Impaci Ithin the first subgroup, less the Impact within the second subgroup. 
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APPENDIX C: SUPPLEMENT TO CHAPTER 4 

NONEXPERIMENTAL ANALYSIS OF THE ASSOCIATION 
ftFTWRfflN f;nT J FCT^GOING AND RECEIPT OF CAREER SERVICES 

I. IntrodMCtton 

The body of this report concerned the experiment conducted to evaluate the impacts 
of assignment to Career Beginnings. Chapter 2 explained the recruitment of a group of high 
school juniors, who were then randomly assigned to two groups - the experimental and the 
controls. The chapter showed that there were no systematic overall differences between the 
two groups in measured (and presumably unmeasured) characteristics expected to be associated 
with college-going. Chapter 3 showed that, while IS to 20 percentage points more 
experimentals than controls received some of the core services provided by Career Beginnings 
(including mentoring, preparation for college entrance exams, and workshops on studying and 
test-taking), a substantial proportion of the control group also received these career services, 
mainly from sources outside the Career Beginnings program. Chapter 4 showed that the higher 
"dose" of career services available to Career Beginnings experimentals led to a 4.7 percentage 
poiP' increase in their rate of attendance td two- or four-year colleges. 

While Chapters i-4 dealt with comparisons between experimentals and controk, this 
analysis takes a step back firom the experiment to look at the much wider variation within each 
group, concentrating especially on the association between service receipt and college-going. 
The analysis starts by explaining why it may be useful to investigate these nonexperimental 
comparisons, and the cautions necessary for interpreting them. Next, the stage is set for 
analyzing the link between services and outcomes. The discussion shows that the relationship 
between service receipt and outcomes is not a simple one: Those who received services had 
different characteristics from those who did not, so any analysis of the link between service 
receipt and outcomes must take account of this. Then follows a comparison of college-going 
among experimentals and among controls who received no services, a few services, and many 
services. The analysis concludes with tests of a few simple hypotheses about service receipt and 
outcomes: b increased service receipt associated with increased college-going? Are individual 
services associated quite differently with rate of college-going? Does mentoring serve as a 
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caulyit that increases the association of other seivices with college-going? 

n. The Purpose of Thb Nonexpcrim ental AmI vsIs and Qintions In Interpretlnf It 

The Career Beginnings evaluation gives a rigorously defensible answer to the question 
of whether the ready availability of more career seivices leads to more college-going among 
disadvantaged in-school youths. However, another important question for programs targeting 
disadvantaged college-bound youths is whether career services - no matter what the seivices 
- raise their college attendance rates. Even from a purely descriptive standpoint, it is of 
interest to know what the rate of college-going was for those who received no career seivices 
compared to the rate for those who got some help, and what the rate of college-going was with 
and without each of the core seivices provided by Career Beginnings and other sou..^?s. 

Unfortunately, the general relationship between sendee receipt and college-going cannot 
be established with anything like the confidence with which it was possible to answer the 
question addressed by the experimental evaluation (the impact of assignment to Career 
Beginnings services in particular). While there was a great deal of variation in service receipt 
and in college-going wUhin the control group and wUhin the experimental group - which is 
necei<*^iy for exploring this more general question about services and college-going - the 
differences in characteristics of those receiving and not receiving services makes it impossible 
to confidently attribute differences in college-gping rates to differences in services alone. 
Compared to the evidence presented in the body of the report, the results presented here 
are much more specdatrve; they cannot answer questions about the unknown true causal 
relationship between career services and college-going. 

As explained in Chapter 2 and Appendix A, the reason these analyses are not at all 
as convincing as experimental analyses is selection bias: Since non-Career Beginnings services 
were not assigned to sample members at random, measured and/or unmeasured characteristics 
of those who got different services were not the same. Thus grouping sample members by the 
kinds or amounts of career services they received and then comparing average rates of college- 
going across the groups would yieU biased estimates of the effects of services. While this 
discussion applies techniques for taking into account differences in measured characteristics 
between r vnple members who got different amounts and combinations of services, it is not easy 
to control for differences in unmeasured characteristics. For example, when comparing college- 
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going rates for those who had intensive classes in reading or math with rates for those who did 
not, it would not be correct to attribute the difference in college-going solely to the reading 
or math classes. Dearly, those who went into such classes might have been systematically 
different from those who did not in characteristics not measured - such as reading or math 
achievement Because groups getting different services may have different unmeasured 
characteristics, we do not know v^ether using a service leads to college-going, or whether the 
college-bound tend to make sure to get the service, or whether someone else tends to make 
sure they get it For this reason, the results presented in this appendix are called "associations" 
or "relationships," and not "impacts" or "effects." 

III. Characteristics of Collcy-Gocrs and Non-Collcge-Gocrs 

To analyze the relationship between college-going and service receipt, the influence on 
college-going of factors other than services received must be taken into account While some 
factors influencing college-going (such as career aspirations and college aptitude) were not 
measured for the Career Beginnings evaluation, many others were. 

Chapter 4 (and especially Table 4.4) addressed the question of how the impact on 
college-going varied among subgroups of the sample, such as men and women. Another 
question to consider is, within the experimental group and within the control group, what 
were the measured characteristics of those who did and did not go to college? Tables C.1 and 
C^ compare the distributions of characteristics of attenden and nonattenders in each group. 
Table CI, for the experimental group, shows that the distribution of virtually every measured 
baseline characteristic varied signiflcantly between those who went to college and those who 
did not 

The proportions for "Both Groups" in the next-to-last column are based on the sample 
sizes in the Grst column. The second and third columns show the characteristics for 
experimentals who did ("Yes") and did not ("No") attend college. Thus, the first two rows of 
the table show that 66.3 percent of experimental members of the impact sample ("Both 
Groups") are women, and 33.7 percent are men. The second and third columns in the body 
of the table show that 69.0 percent of college attenders were women and ^1.0 percent were 
men, and that 63.4 percent of nonattenders were women and 36.6 percent were men. The 
fact that 69.0 percent is more than 66.3 percent shows that those who went to college were 
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TABLE C.l 



EXPERIMENTALS' COLLEGE ATTENDANCE. BY SELECTED BASELINE CHARACTERISTICS 



vniricxvrisxic Arra duu^roup 


dampie 

€4 «A 

di ze 


Ever Attended a 2- 
or 4-Year College* 
June 1988.Nav 1989 
Yes No 

(%) (%) 


DOvn 

Groups 

w 


ntt 

P 


Caw 












FiMit 


412 


69.0 


63.4 


66.3 


0.162 


ml t 


209 


31.0 


36.6 


33.7 




Sitt 












inv DrOfiX^ N6w TOfK 


eZ 


11.7 


14.9 


13.2*** 


A AAA 

0*000 


u€rjr» inOiinQ 


on 


17.8 


10.8 


14. 5 




inoitntpoi is» Indiana 


IIZ 


9.2 


27.8 


18.0 




Jacksonville* Florida 


84 


14.4 


12.5 


13.5 




Rochtster» New York 


81 


11.0 


15.3 


13.0 




danva iina^ LaiiTomia 


M 


18.7 


7.8 


13.5 




Youngstown* Ohio 


So 


17.2 


10.8 


14.2 




Employed anytime during the 












year before randOiA assigi^vnent 












Yes 


332 


50.0 


57.2 


53.5* 


A AAA 

0.082 


NO 


289 


50.0 


42.7 


46.5 




Recent grade average 












A or D 


Z95 


60.4 


33.2 




A AAA 


u» l/» or r 


3ZD 


39.6 


66.8 






ever a scnooi aropoux 












for a semester or more 












Yes 


7 


0.3 


2.0 


1.1* 


0.098 


NO 


Dl4 


99.7 


98.0 






Problems with English and 












another language Is usually 












spoken at home 












Yes 


23 


5.2 


2.0 


3.7* 


0.060 


No 


598 


94.8 


98.0 


96.3 




Living with both parents 












Yes 


269 


46.6 


39.7 


43.3* 


0.095 


No 


352 


53.4 


60.3 


56.7 
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TABLE C.l (continued) 



Chtrtcttristic and Subgroup 


Sample 

Size 


Ever Attended a 2- 
or 4-Year College. 

Jun* 1988-Mav 1989 
Yes No 
(%) (%) 


8oth 
Groups 
(%) 




Has own child (ran) 
Yts 
No 


12 
609 


1.5 
98.5 


2.4 

97.6 


1.9 
98.1 


0.641 


Evar In a summer work 
prograa 

Yas 

No 


251 
370 


32.5 
67.5 


49.2 
50.8 


40.4*** 

59.6 


0.000 


Employad anytime during 
the previous school year 

Yes 

No 


196 
425 


28.2 
71.8 


35.3 
64.7 


31.6* 
68.4 


0.072 


Ever expelled or suspended 
Yes 
No 


119 
502 


11.0 
89.0 


28.1 
71.9 


19.2*** 
80.8 


0.000 


Ethnicity 
White. non-Hispanic 
Black. non-Hispanic 
Hispanic 
Anerlcan Indian 
Asian or Pacific Islander 
Other 


49 

439 
8S 
1 
4S 


7.4 
66.6 
14.1 

0.0 
11.3 

0.6 


8.5 
75.3 
13.2 
0.3 
2.7 
0.3 


7.9*** 
70.7 
13.7 

0.2 

7.2 

0.3 


0.001 


Family receiving cash welfare 
Yes 
No 


191 
430 


28.5 
71.5 


33.2 
66.8 


30.8* 
69.2 


0.239 


Age 18 or older 
Yes 
No 


58 

563 


9.2 

90.8 


9.5 
90.5 


9.3 
90.7 


1.000 


Courses mostly college 
preparatory 

Yes 

No 


215 
406 


47.5 
52.5 


20.3 
79.7 


34.6*** 
65.4 


0.000 


Parent (s) graduated from or 
attended college 

Yes 

No 


193 
428 


38.3 
61.7 


23.1 
76.9 


31.1*** 
68.9 


0.000 
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TABLE C.l (contlnuAd) 



Ch«r«cttr1 sti c and Subgroup 


Sanpla 
SI za 


Evar Attandad a 2- 
or 4-Yaar Collaga. 

Juna 198S-Nav 1989 
Yas No 

(%) (%) 


Both 
Groups 
(*) 




FMlly rtctlving Food Stmps 












YtS 


1S4 


21.8 


28.1 


24.8* 


0.082 


No 


467 


78.2 


71.9 


75.2 




Evtr In • yocr-round training 












progrm not sponsorad by tha 












sMpI* MMbar's high school 












Yas 


42 


8.3 


5.1 


6.8 


0.154 


No 


579 


91.7 


94.9 


93.2 





SOURCES: MDRC calculations from Career Beginnings enrollment form and 1989 survey 

data* 



NOTES: Calculations for this table used self -reported data for all 621 
experlmentals Mho responded to both the 1988 and 1989 surveys* Subgroup sample sizes 
for some characteristics m^ fall short of the total number of sample members because 
of Items missing from data for some survey responders. 

Because of rounding, distributions may not total exactly 100.0 percent. 

*The column labeled "p" Is the statistical significance level of the 
difference between the distributions of characteristics for college attenders and 
nonattenders: that Is, p Is the probability that distributions are different only 
because of random error. A Pearson chl -square test was used to test the hypothesis of 
equal distributions. Statistical significance levels are Indicated as - 1 percent 
or less; ■ 5 percent; ♦ ■ 10 percent. 
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TABLE C.2 



CONTROLS' COLLEGE AHENDANCE. BY SELECTED BASELINE CHARACTERISTICS 



Ch«r«cttr1st1c and Subgroup 


S;<np1t 
Sizt 


Evtr Atttndtd a 2- 
or 4-Ytar Colltgt, 

June 19884(av 1989 
Yts No 

(%) (*) 


Both 
Groups 
(%) 




Stx 

FtMit 

Halt 


379 
233 


61.1 


62.7 
J/ . J 


61.9 
38.1 


0.751 


Sitt 

Tht Bronx, Ntw York 
Gary, Indiana 
Indianapolis, Indiana 
Jacksonvlllt, Florida 
Rochtsttr, NtM York 
Santa Ana, California 
Youngstowi, Ohio 


88 
89 
100 
69 
93 
78 
95 


14.6 

8.3 
12.3 
16.3 
17.6 
Id* J 


14.1 

24.1 

10.3 
14.1 
8.0 

1 C A 


14.4*** 

14.5 

16.3 

11.3 

15.2 

12.7 

15.5 


0.000 


Employtd anytlm during the 
ytar btfort random assignment 

Yts 

No 


314 
298 


45.8 
54.2 


56.6 
43.4 


51.3*** 
48.7 


0.010 


Rtctnt grade average 
A or B 
C, 0, or F 


300 
312 


61.8 
38.2 


36.7 
63.3 


49.0*** 
51.0 


0.000 


Ever a school dropout 
for a semester or more 

Yes 

No 


7 

605 


0.0 

100.0 


2.3 
97.7 


1.1** 
98.9 


0.025 


Problems with English and 
anothtr language Is usually 
spoken at hone 

Yts 

No 


23 
589 


5.0 

95.0 


2.6 
97.4 


3.8 
96.2 


0.175 


Living with both partnts 
Yts 
No 


259 
353 


46.8 

53.2 


37.9 
62.1 


42.3** 
57.7 


0.032 
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TABLE C.2 (contlnutd) 



Characttrlttlc and Subgroup 



SMpIt 
SIZt 



Evtr Atttndtd « 2- 

or 4-Yt«r Col lift, 

Juno igia4tov 1M9 Both 
Yos No Groups 

(%) (%) (%) 



Has own chlld(ron) 



Ytt 


16 


1.3 


3.9 


2.6* 


0.088 


No 


S96 


98.7 


96.1 


97.4 




Evtr In « suMRtr work 












progrM 












Yts 


244 


30.9 


48.6 


39.9**» 


0.000 


No 


368 


69.1 


51.4 


60.1 




Enploytd anytlnt during 












tho orovlout school voar 












Yos 


188 


28.6 


32.8 


30.7 


0.296 


No 


424 


71.4 


67.2 


69.3 




Evor oxpollod or suspended 












Yos 


108 


9.6 


25.4 


17. 6*** 


0.000 


No 


504 


90.4 


74.6 


82.4 




Ethnicity 












Whitt, non-Hi spanic 


65 


11.6 


9.6 


10. 6** 


0.016 


Black, non-H1sp«n1c 


410 


62.5 


71.4 


70.0 




Hispanic 


94 


16.3 


14.5 


15.4 




AmH can Indian 


2 


0.0 


0.6 


0.3 




Asian or Pacific Islander 


41 


9.6 


3.9 


6.7 




Othtr 


0 


0.0 


0.0 


0.0 




Fanlly rtcelving cash wtlfare 












Yts 


192 


28.9 


33.8 


31.4 


0.227 


No 


420 


71.1 


66.2 


68.6 




Age 18 or older 












Yts 


66 


10.6 


10.9 


10.8 


1.000 


No 


546 


89.4 


89.1 


89.2 




Courses aostly college 












preparatory 












Yes 


234 


51.2 


25.7 


38. 2*** 


0.000 


No 


378 


48.8 


74.3 


61.8 




Parentis) graduated frcoi or 












attended college 












Yes 


212 


39.5 


29.9 


34. 6** 


0.016 


No 


400 


60.5 


70.1 


65.4 
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TABLE C.2 (continued) 



Ch«racttr1stic and Subgroup 


Sanplt 
Sizt 


Evtr Att«nd«d • 2- 
or 4-YMr Co11«9«. 
Jun. 19a84Uv 1M9 
Yts No 

(%) (%) 


Both 
Groups 
(%) 


P* 


FMlly rtctlving Food Stinips 
No 


163 
449 


22.6 
77.4 


30.5 
69.5 


26.6** 
73.4 


0.033 


Evtr In a ytar-round training 
progran not sponsortd by tht 
tanpit naabtr's high school 

Yts 

No 


58 
554 


10.0 
90.0 


9.0 
91.0 


9.5 
90.5 


0.788 



SOURCES: HORC calculations from Carter Beginnings enrollment form and 1989 survey 

data. 



NOTES: Calculations for this table used self-reported data for all 612 controls 
Mho responded to both the 1988 and 1989 surveys. Subgroup sample sizes for some 
characteristics may fall short of the total number of sample members because of Items 
missing from data for some survey responders. 

Because of rounding, distributions may not total exactly 100.0 percent. 

*The colum labeled *p** Is the statistical significance level of the 
difference between the distributions of characteristics for college attenders and 
nonattenders: that Is, p Is the probability that distributions are different only 
because of random error. A Pearson chl -square test was used to test the hypothesis of 
equal distributions. Stat1sv:cal significance levels are Indicated as • 1 percent 
or less; • 5 percent; ♦ • 10 percent. 
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jDXxt likely than nonattendeis to be women, though the p value of 0.162 shows that this 
difference was not statistically significant Similarly, the other rows of the table show that 
those who went to college were more likely to have entered the sample at Gary, Jacksonville, 
Santa Ana, or Young^town; less likely to have worked during the base year or during the 
school year; tess likely to have been in a summer work program; much more likely to have had 
good grades; more likely to have lived with both parents; less likely to have been expelled or 
suspended; less likely to come from welfare homes; much more likely to have taken mainly 
college-preparatory courses; and more likely to have parents who attended college. Table C.2 
shows similar results within the control group. 

IV, The RciattonshiD Between Characteristics and Services Received 

Measured and unmeasured characteristics of sample members affected not only their 
college-going but also their rates of receipt of various services. Chapter 3 (and especially Table 
3.11) showed the association between various measured characteristics and experimental-control 
differences in average number of services received Tables C3 and C4 address this same issue 
in a slight^ different way. Each column of the table reports the results of a linear regression 
of receipt of a particular service on measured baseline characteristics. The entry for the 
**ConstanC or intercept, is the overall rate of receipt of the service. For example, 20.1 percent 
of ejq)erimentals and 13.7 percent of controls received help with reading or math. The other 
coefficients show the association of each baseline characteristic with receipt of the service, 
controlling for the influences of all other measured baseline characteristics. For example, the 
entry for "Male" in the first column of Table C.3 shows that, other measured characteristics 
held constant, experimental group men were 4.2 percent more likely to get special help with 
reading or math than experimental group women. The corresponding entry in the first column 
of Table C4 shows that, other measured characteristics held constant, control group men were 
9.0 percent more likely to get special help with reading or math than control group women. 

These tables tell an interesting story about the kinds of services received by different 
kinds of experimentals and different kinds of controls. Jacksonville and Bronx experimental, 
all other measured charaaeristics held constant, were much more likely to get reading or math 
help than experimentals in other sites, bu^ Bronx controk were also much more likely than 




TABLE C.3 

ASSOCIATION OF SERVICE RECEIPT WITH SELECTEO BASELINE CHARACTERISTICS AMONG EXPERIMENT ALS: 
ESTIMATED REGRESSION COEFFICIENTS FOR PROBABILITY OF RECEIVING SERVICES 



Rtfrtsftor or SUtlttIc 


Reading or 
Math Nelp 


Tutoring 


Study 
Skills 


Entrance 
Exaa Help 


Application 
For* Help 


College 
Fair 


Career 
Fair 


Job Skills 
Workshop 


Mentoring 


Counseling 
Sessions 


Faally 
Planning 


ContUnt 




17. 6*** 


42 .O*** 


55. 9*** 


63.9*** 


60.1*** 


56.5*** 


68.9*** 


64.3*** 


75.0*** 


23.8*** 


Milt 


4.2 


11. I*** 


2.1 


.4.5 


.3.8 


-4.9 


.1.1 


2.7 


.6.6 


1.9 


1.4 


Sttt 

Tilt Brtnx* Nt« York 
Gtfy» ItdttM 
Mtaiit^ltftt IndUiM 
JtcktORvtllt* FloHdt 
Rtdittttr» Nt« York 
SmU Am, CtlHomU 
Ytwiftttiiii, Ohio 


16. 4** 

7.0 

40.9*** 

-0.1 
6.0 
0.4 


I2.9** 

•1.7 

27. 5*** 
.5.9 
1.5 
•4.9 


13.6* 
33. 5*** 

22.2*** 
•2.2 
8.6 

31. 6*** 


21. 0** 
29.1*** 

8.0 
.0.9 
14.0 
11.4 


.13.4* 
0.8 

1.2 
.6.4 

9.1 
12.0 


21.7*** 
4.1 

.3.2* 
4.4 

15.2 
.8.6 


2.5 

23.0*** 

21.3*** 
18.3** 
22.3** 
23.1*** 


.23.1*** 
25.0*** 

6.7 
3.2 
3.4 
14.7* 


.12.0 
7.1 

12.0 
24.1*** 
12.3 
18.3** 


9.5 

20.8*** 

22.2*** 
19.1*** 
10.4 
8.4 


8.7 
15.2** 

.3.8 
.10.7 
.5.2 
6.3 


Enploytd tnytlM during tht 
yttr ktfort randov tttlgiiMfit 


2.6 


.8.8** 


5.1 


2.2 


1.1 


3.3 


6.3 


-2.7 


.8.3 


10.3** 


-3.5 


Rtctfit fradt tvtrtgt 

C. D« or F 


.8.2** 


•4.1 


-11.8*** 


.10.0** 


.5.6 


.9.6** 


.0.2 


-8.5** 


.0.1 


.11.2*** 


0.2 


Evtr t school dropout 
for t ttMtttr or Mort 


.23.2 


.10.2 


.15.3 


1.3 


.19.7 


.4.0 


17.9 


.1.3 


-24.8 


2.8 


14.3 


ProhltM «tth English and 
anothtr lanfuagt Is usually 
spoken at hone 


4.7 


.4.0 


.5.8 


.4.3 


.6.4 


.4.9 


-23.7** 


-9.1 


6,3 


-2.6 


13.0 


Living with both parents 


.7.3** 


.2.2 


.7.7* 


.1.3 


8.7** 


.1.1 


.0.3 


2.9 


.2.7 


.3.1 


-3.9 


Has oim €h11d(ren) 


1.5 


.9.6 


.16.9 


.0.1 


7.4 


.7.0 


10.7 


.12.4 


21.4 


.25.2** 


-4.6 


Ever fn a sunaer irork 
progran 


0.3 


5.8 


3.2 


8.3 


7.6 


.2.3 


9.1* 


3.3 


9.6* 


3.6 


4.4 


Enployed anytime during 
the previous school year 


.2.1 


5.0 


.8.1 


.3.1 


.2.1 


5.0 


.10.1* 


-4.0 


-2.4 


-5.2 


0.8 


Ever expelled or suspended 


.0.6 


.3.7 


2.5 


.4.2 


.7.1 


.1.6 


3.0 


-7.7 


0.0 


-0.6 


7.1 
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TABLE C.3 (continued) 



Rt«d1n9 or Study Entranct Application Colltge Career Job Skills Counseling Fanlly 

Htfrttior er Statistic Math Ntip Tutorlnv Skills Urn Help Fom Help Pair Pair Workshop Mentorinf Sessions Planning 



Etfciitctty 



Mittt, nen^Nttpanlc 


-8.8 


-1.0 


-31. 7*** 


-20.8*** 


-34.0*** 


-21.6*** 


-22.8*** 


-23.4*** 


-14.8** 


-3.0 


-12.5* 


— * — — » - 
mipMiic 


.ft 


-1 6 


-18.4*** 


-10 4 


-5.4 


-2.3 


9.0 


3.8 


-7.1 


0.6 


-1.0 


•lack or ether 
























Faatly rtcelvtnf cash welfare 


-3.3 


-5.3 


-4.2 


-2.8 


5.1 


-3.8 


-4.1 


-0.5 


-9.0* 


-4.7 


-8.1* 


Afe to or eioer 


10 3* 


II 1 


-1 3 
— I . «f 


-7 7 


-5 7 


-4 8 


-6.3 


0.4 


-0.4 


-11.8* 


1.8 


raimea ■ettlv collede 


















8.8* 


1.7 




preparatory 


-3.4 


3.2 


-3.6 


10.6** 


7.3 


10.8** 


13.1*** 


1.5 


3.5 


rareni|S| f reouaiee rrov or 
























attended coiiefe 


A II 


-7 7** 


0 ft 


3. C 


-3 7 




1.0 


-0.5 


-0.1 


3.3 


2.2 


raMiiy rtceivinf rood staaps 


J. 1 


n ft 
II. Q 




.1 *i 
-1.9 


.0 ^ 


1 Q 


-4.9 


1.0 


-1.2 


-0.3 


-1.2 


Ever in a yeer-rouna training 
























profran net ipentoreo oy tne 




















8.5 






2.5 


-11.0* 


-1.6 


13.2* 


5.2 


16.0** 


5.6 


-1.2 


6.0 


3.9 


Mber of eheervatlons 


821 


621 


621 


621 


621 


621 


621 


621 


621 


621 


621 


Ihflber of experlMOtals 


821 


621 


621 


621 


621 


621 


621 


621 


621 


621 


621 


Nunlber of controls 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


Oegroes of f rtedea for error 


596 


596 


596 


596 


596 


596 


596 


596 


596 


596 


596 


Error nean sfoaro 


1409.58 


1310.63 


2150.75 


2257.48 


2158.60 


2250.43 


2335.02 


1975.04 


2167.88 


1821.83 


1773.80 


R sipiare 


0.16 


0.13 


0.15 


0.12 


0.10 


0.10 


0.09 


0.11 


0.09 


0.07 


0.06 


Mean of dependent variable 


20.1 


17.6 


42.0 


55.9 


63.9 


60.1 


56.5 


68.9 


64.3 


75.0 


23.8 


P statistic 


4.68 


3.74 


4.48 


3.43 


2.81 


2.75 


2.40 


3.23 


2.58 


1.77 


1.65 


f value of P statistic 


.00 


.00 


.00 


.00 


.00 


.00 


.00 


.00 


.00 


.01 


.03 



SOURCES: MWC calculations fro« Career Beginnings enrol Iwent fons and 1988 survey data. 

NOTES: The dependent variable In each regression equation ms 100 for each experimental who received the service and zero othenvlse. Each 
characteristic on the right-hand aide of each equation was measured as a deviation from Its mean; thus, the coefficient of the constant Is the 
percentage of the sample that received the service. 

A too-tailed t-test vat applied to each coefficient estimate. Statistical significance levels are Indicated as *** - 1 percent or 
less; ** ■ 5 percent; * ■ 10 percent. 

The p value of the P statistic Is the probability of obtaining these coefficient estimates If the true chance of receiving the service 
did not vary with any characteristic. Thus, the closer the p value Is to zero, the more Important are differences In characteristics between those 
O JO did and those who did not receive the service. 



TABLE C.4 



ASSOCIATION OF SERVICE RECEIPT WITH SELECTED BASELINE CHARACTERISTICS AMONG CONTROLS: 
ESTIMATED REGRESSION COEFFICIENTS FOR l»ROBAdILITY OF RECEIVING SERVICES 





OlAAiil IfeM AV> 

IIMViwig Vw 






Entrance 


Application College 


Career 


Ink Cblll* 
JOD 5X1 1 It 




tountei ing 


Faaliy 




nam neip 


lu^onng 


«X1 lit 


txan neip 


Cam M*1b 




rair 


worxtnop 


nenioring 


Settlont 


Pi anning 


COWttAUt 






Z4.Z"** 


39.5""" 




56. Z*** 


41.Z""" 


56.4""* 


45. 


00. Z""" 


16. 5*** 


tkit 


A AAA* 


3.Z 


3.2 


3.6 


-3.2 


-5.2 


1.0 


-4.8 


-4.7 


1.8 


-0.9 


S1U 
























Tht Bronx, Nt« York 




14. 


32.6*** 


12.0 


0.3 


24. 5*** 


27. 4*** 


1.0 


14. 7^ 


17. 9** 


2.1 


Gory, IMIam 


0.2 


3.0 


Z3.3 


9 a 

Z.8 


.5.4 


3.5 


9C 1**# 


13.3* 


18. 6 " 


4.6 


4.0 


IndlMiopollt, IndUiM 
























Jacktonvnit, FloHdo 


8.2 


14. 6*** 


17. I** 


-1.8 


-10.4 


0.4 


25. 2*** 


-6.4 


16.3* 


1.2 


13.0* 


Nocnooitrg ntv lortc 


1 a 




0.5 


1 Q 
-l.V 


-2.3 


2.2 


in ^ 

10.3 


9 7 
Z. / 


^ 9 


1ft ft** 

la.!)" 


3.0 


SonU Am, CollfomU 


8.3 


4.3 


-0.3 


.17.5* 


0 9 
•o.Z 


-6.1 


29. 3*** 


-9.9 


-20. 3** 


-6.3 


1.6 


YoMifttowi, Ohio 


1.0 


-8.0 


9.7 


-10.7 


-6.0 


-9.8 


11.9 


7.0 


6.3 


-5.8 


-6.5 


Eiplojftd anytlM during the 
























yoar btfort r«ndo« utIgraMnt 


-4.5 


1.3 


0.6 


-6.5 


in J** 


-1.0 


3.7 


-4.0 


4.0 


4.3 


-3.6 


Rtcont 9^*^ Avtragt 
























C, 0, or F 


1.1 


.2.4 


2.0 


.7.6* 


-8.2* 


-14.2*** 


-6.0 


-2.2 


1.2 


-5.2 


2.9 


Evtr « school (^ropoyt 
























for « toMf tor or nort 


10.9 




11.0 


3.4 


-7.1 


13.5 


40.9** 


27.7 


-13.4 


12.5 


8.1 


froblm with EnglUh and 
























•fwthtr lanouagt It usual 1y 
























tpokofi at ho«t 


-2.7 


.3.1 


2.3 


-4.1 


lo.D 


-/ .0 


-7.0 


2.4 


0.3 


-9.1 


-6.7 


LI VI 119 w\\n ooin paronit 


- 1 


ft a** 


1 Q 

1 .7 


n 1 
-U. 1 


-0.7 


3.8 


n ft 

-11.3 


1 ft 


9 9 


ft 1* 

0.1 


n 7 
-u. / 


Hat owi €Mld(ren) 


.9.6 


-3.4 


-14.0 


-24. 9** 


13.3 


-8.2 


-8.8 


-15.8 


O.J 


-15.2 


1.4 


Evtr In a tuMwr work 
























progran 


5.8 


2.3 


8.4* 


1.4 


15.8*** 


1.3 


4.8 


7.9 


6.8 


-6.5 


0.9 


Eiploytd anytiM during 
























tht pro V lout tchool year 


1.^ 


.3.5 


0.9 


2.4 


-0.0 


-2.4 


-8.4» 


5.2 


0.3 


-10.5** 


2.2 


Ever expelled or tutpended 


-3.2 


-0.3 


-3.8 


-4.9 


-0.8 


.7.2 


6.8 


-3.4 


6.0 


4.8 


2.9 
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TABLE C.4 (continued) 



RMdIng or Study Entrance Application College Career Job Skills Counseling Family 

War— sor »r Statistic Math Help Tutoring Stcllls Exaw Help Fom Help Fair Fair Works hop Mentoring Sessions Planning 

EtkRlclty 

Nhtte. ROii-Nlspaiilc -5.0 1.5 -4.9 -10.9 0.4 -7.4 -6.7 6.6 -6.7 7.3 4.4 

Mtpanfc -4.3 7.0 4.2 1.7 4.1 9.1 -1.8 6.6 0.8 -9.2 5.2 

Hack or otiier — — 

Fa«ny receiving cash welfare 0.8 1.6 2.6 -2.3 -6.4 9.3* -2.9 1.0 S.O 10.2* -1.8 

Age 18 or older 5.0 8.0* 8.3 16.5** 3.9 -1.7 5.2 -1.8 1.0 -10.8* 1.5 

Coursos aostly college 

prenrttory 2.7 -1.3 1.4 8.7** -5.2 17.1*** l.l -7.4* -2.8 9.3** -1.7 
* f arenC 0 f r ed ua ted f ro« or 

attended college -1.6 1.2 0.5 6.5 -0.4 4.4 -5.4 -10.5** 0.1 -1.9 5.6* 

I Faally receiving Food Stamps -2.4 6.4* -1.9 -1.7 -1.1 -8.8 2.4 -9.3 -0.4 -6.8 5.3 

tt Ever In a year-round training 
^ progroM not sponsored by the 



snple Miiber*s high school 3.5 1.7 15.9*** 0.7 24.2*** 1.8 24.6*** 3.3 13.9** -0.6 3.7 



Iluii6er of observations 612 612 612 612 

Nuaber of experl«enta1s 0 0 0 0 

Nuiiber of controls 612 612 612 612 

Degrees of freedom for error 587 587 587 587 

Error seen square 1079.87 1060.64 1736.22 2288.04 

R sijuare 0.13 0.08 0.09 0.08 

Mean of dependent variable 13.7 12.9 24.2 39.5 

F statistic 3.50 2.07 2.47 2.19 

r value of F statistic .00 .00 . 00 .00 



612 612 612 612 612 612 612 

0 0 0 0 0 0 0 

612 61? 612 612 612 612 612 

587 58> 587 587 587 587 587 

2420.28 2219.65 2295.74 2444.62 2389.93 2139.42 1384.67 

0.07 0.14 0.09 0.05 0.07 0.08 0.03 

52.8 56.2 41.2 56.4 45.1 66.2 16.5 

1.80 3.82 2.45 1.20 1.96 2.22 0.92 

.01 .00 .00 .24 .00 .00 0.58 



SOUHCES: MDRC calculations fro« Career Beginnings enrollment form and 1968 survey data. 

NOTES: The dependent variable In each regression equation was 100 for each cdntrol who received the service and zero otherwise. Each 
characteristic on the right-hand side ef each equation was measured as a deviation from Its mean; thus, the coefficient of the constant Is the 
percentage of the sample that received the service. 

A two-tailed t*test was applied to each coefficient estimate. Statistical significance levels are Indicated as *** - 1 percent or less; 
*• • 5 percent; • ■ 10 percent. 

The p value of the F statistic Is the probability of obtaining these coefficient estimates If the true chance of receiving the service 
did not vary with any characteristic. Thus, the closer the p value Is to zero, the more Important are differences In characteristics between those who 
and those who did not receive the service. 



controls at other sites to get this kind of help, and even more likely to get it than Bronx 
experimentals. 

The strengths of the programs at Gary and Jacksonville are borne out by the 
coefficients for those sites in Table C.3, which for the most part are large both absolutely and 
in comparison with the control group counterparts (Table C.4). Among experimental and 
among controls, those with poor grades at baseline were less likely to get help with college 
application forms or entrance exams and less likely to attend college fairs. They may have 
been poor students, who were less likely to seek out such services, or perhaps their teachen 
and guidance counselors tended not to recommend that they get such services. In addition, 
among experimentals, but not among controk, those with poor grades were less likely to get 
help with reading or math, study skills workshops, job skills v^rkshops, or counseling. Among 
experimentals, but not among controls, white sample members were significantly less likely to 
get most of the 11 services. In the control group and, to some extent, in the experimental 
group, those who had previously been in programs were more likely to receive some of the 
11 services. 

V, The Associatioa Betwten College«Going and Types of Services Received, 
Controlling for Measured Baseline Characteristics 

The discussion so far has shown that measured baseline characteristics are associated 
with differences in both college-going and service receipt. Table C.5 tries to remove this 
"background noise" in order to present a clearer picture of the relationship between college- 
going and receipt of particular career services. It results from a regression analysis which, at 
the same time, assesses the association of college-going with both measured baseline 
characteristics and services received. It shows that, for the most part, after taking into account 
the ociation with college-going of various measured baseline characteristics, such as sex, 
receipt of a service sometimes has a statistically significant positive association with college- 
going. For example, attending special preparatory classes for college entrance exams is 
associated with an 8.7 percentage point higher rate of college-going among experimentals and 
a 9.0 percentage point higher rate of college-going among controls. 

However, sometimes the association is negative, as in the case of attending family 
plarming workshops. Attending a family planning workshop is associated with a 10.3 percentage 
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TABLE C.S 



ASSOCIATION OF BASELINE mRACTERISTICS AND SERVICES RECEIVED WITH COLLEGE-GOING: 
ESTIMATED REGRESSION COEFFICIENTS AND P VALUES FOR PROBABILITY OF 
ATTENDING A TWO- OR FOUR-YEAR COLLEGE, JUNE 1988-MAY 1969 



Rtgrtssor or Statistic 



ExDtriwentil Stmplt 
Coefflcltnt p* 



Control Sample 
Cotfflcltnt E 



Constant 

Malt 

Sitt 

Tht Bronx, Ntw York 
Gary, Indiana 
Indianapolis, Indiana 
Jacksonville, Florida 
Rochester, New York 
Santa Ana, California 
Youngstown, Ohio 

Employed anytime during the 
year before random assignment 

Recent grade average 
C, D, or F 

Ever a school dropout 
for a semester or more 

Problems with English and another 
language Is usually spoken at home 

Living with both parents 

Has own chlld(ren) 

Ever In a simmer work program 

Employed anytime during 
the previous school year 

Ever expelled or suspended 

Ethnicity 
White, non-Hispanic 
Hispanic 
Black or other 



52.5*** 
-4.8 



11.6 
13.1* 
25. 8*** 
15.0* 



9.4* 



-16.5 
-20.1 

5.1 

1.2 
-4.2 
-11.2** 

-9.8* 
•16.7*** 



-7.3 
-2.7 



0.000 
0.216 



2.4 0.762 
22.1*** 0.002 



0.120 
0.074 
0.002 
0.063 



0.065 

0.000 

0.244 

0.631 
0.755 
0.752 
0.020 

0.053 
0.001 



0.312 
0.663 



49.2*** 0.000 
2.0 0.614 



6.1 
11.0 

12.3 
16.9** 
31.1*** 
6.9 



0.476 
0.140 

0.136 
0.026 
0.001 
0.403 



-2.3 0.637 

-16.4*** 0.000 

-26.1 0.145 

-8.3 0.458 

1.3 0.749 

-9.6 0.415 

-4.5 0.362 

-1.4 0.781 

-13.1** 0.013 



1.0 0.886 
0.8 0.897 



(continued) 
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TABLE C.5 (contlnutd) 



ExDTliimtal Siwplt Control Sawol* 
Rtgrtssor or St«tist1c Cotfi'icitnt p* Cotfficiant p* 





-2.3 


0.636 


1.3 


0.802 


Agt 18 or oldtr 


0.6 


0.930 


2.3 


0.714 


Cour«*« Mstiv collAOft Drtoaritorv 


16. 6*** 


0.000 


18.3*** 


0.000 


P«rtnt(s} graduated from or 


8.5** 


0.041 


5.3 


0.196 


Family rtcelving Food Stamps 


-2.6 


0.614 


-1.8 


0.723 


Evtr In a ytar-round training 
program not sponsored by the 


10.9 


0.137 


1.6 


0.805 


Took classes In reading or math. 

cpciv 1 runi rc^uiQi awnwvi wiQascs 


10.2* 


0.054 


-1.7 


0.777 


Got Individualized academic 

^ii^Aff*1 nn 
VII Vwi 1 nji 


7.2 


0.178 


-9.5 


0.103 


Received help Kith studying 
and test-test 1 no skills 


-3.4 


0.459 


-4.1 


0.427 


Attended classes or workshops to 
Dreoare for coll^ioe entrance exams 


8.7** 


0.048 


9.0** 


0.034 


Attended classes on completing forms 
for adalsslon and financial aid 


-3.8 


0.394 


-0.6 


0.887 


Attended a college fair or other 
college inTonnaxion cveni 


0.8 


0.842 


9.7** 


0.023 


Attended a career fair or other 
Job Information event 


6.3 


0.141 


-2.0 


0.646 


Attended a Job-readiness skills 
workshop on dress, lateness, etc. 


5.0 


0.276 


-8.7** 


0.034 


Had mentor other than relative 
or friend 


3.8 


0.349 


8.7** 


0.030 



(continued) 



-131-1 



TABLE C.5 (contlnutd) 



Rtyrtssor or 9x1x1 sxic 


ExDerlMntal Samole 


Control Samole 


vOVT T 1 V 1 •n \ y 




Ntt it 1t«st twict Mith counstlor 


7 C* A AAA 




Attmdtd « c1«$$ or Mrkshop 






on fMrily planning 


-10.3 O.U£Z 


.7 c n 110 


Numbtr of ob$trv«t1on$ 


621 


6IZ 


til aha r nf ftXIMrlMntAl S 


621 


0 


Hmb%r of controls 


0 


612 


D«f rMs of f rttdom for error 


S85 


576 


Error Man squart 


1920.67 


2026.25 


R squart 


0.2745 


0.2370 


Htan of deptndtnt variable 


52.496 


49.183 


F statistic 


6.32 


5.11 


P value of F statistic 


0.0001 


0.0001 



SOURCES: HDRC calculations from Career Beginnings enrollment form and 1988 and 
1989 survey data. 



NOTES: The dependent variable In each regression equation was 100 for each 
college attender and zero otherwise. Each characteristic and service on the 
right-hand side of each equation was measured as a deviation from Its mean; thus, the 
coefficient of the constant Is the percentage of the overall sample that attended 
college. 

*The p value of the F statistic Is the probability of obtaining these 
coefficient estimates If the true chance of attending college did not vary with any 
characteristic or service. Thus» the closer the p value Is to zero, the more 
Important are differences In characteristics and services received between college 
attenders and nonattenders. A two-tailed t-test was applied to each coefficient 
estimate. Statistical significance levels are Indicated as > 1 percent or less; 
» 5 percent; * » 10 percent. 
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point lower rate of college-going among cxperimentals and a 7.5 percentage point lower rate 
of college-going among controls. It is unlikely that attending the family planning workshop 
caused a kiwer rate of college-going. Rather, those who did not go to college because they 
instead started families may, in high school, have been more likely to attend a family planning 
workshop. There was a probable selection bias on an unmeasured characteristic: "more likely 
to be sexually active." Those who had the unmeasured characteristic were probably more likely 
to get the family planning treatment and less likely to go to college because of the unmeasured 
characteristic rather than because of any effect of family planning workshops themselves on 
college attendance. This example of the selection bias problem illustrates why the results 
presented in this appendix are "associations** or **relationships,** not **impacts** or **eCfects.** 

VI. The Assoclatton Between Collcy-Goin£ and Number of Services Received. 
ControlHng for Measured Baseline Characteristics 

Although, as just shown, there is sometimes a negative association between sendee 
receipt and college-going, a reasonable hypothesis is that, all things considered, receiving more 
services is associated with a higher rate of college-going. Table C.6 shows the results from 
separate linear regressions run for experimental and controls, in which measured baseline 
characteristics of youths and the number of services they received were used as "predictors** of 
college-going. For example, the row labeled "Male** shows that, among experimental, males 
were 4.2 percent less likely to go to college than were females and that, among controls, males 
were 1.5 percent more likely to be college attendees. The row labeled **Number of Services 
Received** shows the association between added services and rate of college attendance. The 
coefGcients in the table show that, on average for experimental, one added service is 
associated with a 2.7 percentage point higher rate of college-going; for controls, it is associated 
with a 13 percentage point higher rate. 

However, Table C7 shows that this average misrepresents what happens at the extremes 
of service receipt Receipt of each of the 11 career services presented in Table 3.6 was tallied 
for each sample member, and the totals were grouped, as shown, into categories: none; 1 or 
2; 3, 4, or 5; and 6 to 11. The 621 experimental and 612 controls in the impact sample were 
analyzed separately. 

As shown in the table, 22.9 percent of the 16 experimental who received none of the 
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TABLE C.6 



ASSOCIATION OF BASELINE CHARACTERISTICS 
AND NUMBER OF SERVICES RECEIVED WITH COLLEGE-GOING: 
ESTIMATED REGRESSION COEFFICIENTS AND P VALUES FOR PROBABILITY OF 
ATTENDIk*: A TWO- OR FOUR-YEAR COLLEGE, JUNE 1988-MAY 1989 



Rtgrtssor or Statistic 


Exoerlmental Samole 


Control Sanit 


Coefficient 


p* Coefficient 




Constant 


52. 5*** 


0.000 


49. 2*** 0.000 


Nunbtr of strvlcts recclvtd (0-11) 




0.000 


1.3 


0.107 


Naif 


-4.2 


0.273 


1.9 


u./uo 


Sitt 










Tht Bronx 1 NtM York 


3.9 


0.615 


O.o 


U.4ZQ 


Gary, Indiana 


22.7*** 


0.001 


9.1 


0.223 


Indianapolis, Indiana 










Jacksonvlllai Florida 


17.7** 


0.014 


9.3 


0.257 


Rochtsttr, Naw York 


16. 6** 


0.022 


18. 0** 


0.020 


Santa Ana, California 


27. 9*** 


0.001 


26. 2*** 


' 0.004 


Youngstowni Ohio 


14.0* 


0.081 


4.9 


0.557 


Employtd anytime during 










tht ytar btfort randon assignment 


9.9** 


0.049 






Recant grade average 










C| D, or F 


.17.4*** 


0.000 


-18. 3**^ 


' 0*000 


Ever a school dropout 










for a semester or more 


-21.4 


A n e 

O.ZIS 




n mo 


Problems with English and another 










language Is usually spoken at home 


3.0 


0.774 


-9.9 


0.386 


Living with both parents 


0.8 


0.836 


2.3 


0.574 


Has own child (ran) 


-4.1 


0.756 


-11.2 


0.350 


Ever In a summer work program 


-10. 9** 


3.023 


-6.8 


0.172 


Employed anytime during 










the previous school year 


-10. 2** 


0.043 


-2.8 


0.570 


Ever expelled or suspended 


-17. 9*** 


0.000 


12.9** 


0.015 


Ethnicity 










White, non-Hispanic 


-4.8 


0.503 


-0.6 


0.926 


Hispanic 


-1.9 


0.759 


-1.4 


0.831 


Black or other 











(continued) 
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TABLE C.6 (continued) 





ExDsrlMfital SaMola 


Control Sai 




Rtgmsor or Statist 1c 


Cotfflcltnt 


P* 


Cotfflcltnt 




FmI 1 V rftcftlvlmi caih * jlfarA 


•2.4 


0.621 


2.7 


0.600 


AoA IB or oldor 


0.8 


0.902 


0.5 


0.939 


Courstf mostly colltgt prtptrttory 


17.0*** 


0.000 


22.3*** 


0.000 


rimii^s^ yrcauciM Tron or 
*tttndtd colltgt 


8.0* 


0.053 


6.5 


0.110 


Family rtctlving Food Stamps 


•2.6 


0.614 


•3.7 


0.491 


£vtr In a ytar-round training 
progran not iponsortd by the 
samplt manNr's high school 


11.0 


0.132 


-0.2 


0.982 



Nuabtr of obstrvatlons 


621 


612 . 


Nunbtr of txptrlmtntals 


621 


0 


Nunbtr of controls 


0 


612 


Otgrtts of frttdom for error 


595 


586 


Error Man squart 


1943.28 


2106.80 


ft squart 


0.2534 


0.1929 


Ntan of dtptndtnt variable 


52.496 


49.183 


F statistic 


8.08 


5.60 


P value of F statistic 


O.OOOl 


0.0001 



SOURCES: MDRC calculations from Career Beginnings enrollment form and 1988 and 
1989 survey data. 

MTES: The dependent variable In each regression equation was 100 for each 
college attender and zero othennflse. The number of services and each characteristic 
on the r1ght'*hand side of each equation was measured as a deviation from Its mean; 
thus» the coefficient of the constant Is the percentage of the overall sample that 
attended college. 

*The p value of the F statistic Is the probability of obtaining these 
coefficient estimates If the true chance of attending college did not vary with any 
characteristic or service. Thus, the closer the p value Is to zero, the more 
Important are differences In characteristics and In number of services received 
between college attenders and nonattenders. A two-tailed t-test was applied to each 
coefficient estimate. Statistical significance levels are Indicated as 
*^ ■ 1 percent or less; ■ 5 percent; ♦ ■ 10 percent. 
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TABLE C.7 



EDUCATIOtML OUTCOMES FOR EXPERINENTALS AND CONTROLS. 
BY NUMBER OF SERVICES RECEIVED 



OutcoM and Nunbtr of 
Sttrvlcts Rtctlvtd 


ExptHmtnttls 


P* 


Samplt 

Silt 


Controls 


P* 


Samplt 

Size 


Evtr atttndtd • 2* or 4-yt«r 














colltgt, JiiM 1968-May 1989 (%) 














Roctlvtd no ttrvlcts 


22. 9*** 


0.006 


16 


27.6* 


0.084 


27 


Roctlvtd 1 or 2 strvlcts 


46.5 




90 


46.7 




145 


RMAlvAd 3. 4. or S s*rv1cAs 


49.3 




201 


50.5 




250 


RtctlvMl 6 or wrt ttrvlcts 


57.8 




314 


52.4 




190 


Nuabtr of nonths ttttndtd 














a 2- or 4-yttr colltgt. 














Junt 1988-ltay 1989 














Rtctlvtd no ttrvlcts 


2.2*** 


0.005 


16 


2.3* 


0.078 


27 


RtctiVty 1 or £ strvicts 












1 AC 


Rtctlvttd 3| 4| or 5 strvlcts 


4.2 




201 


4.3 




250 


Rtctlvtd 6 or nort strvlcts 


5.1 




314 


4.5 




190 


Atttndtd • 2- or 4-yttr 














colltgt during Nay 1989 (%) 














Rtctlvtd no strvlcts 


22. 4** 


0.032 


16 


27.6 


0.250 


27 


Rtctlvtd 1 or 2 strvlcts 


45.7 




90 


41.8 




145 


Rtctlvtd 3, 4, or 5 strvlcts 


43.2 




201 


44.7 




250 


Rtctlvtd 6 or nort strvlcts 


51.8 




314 


47.0 




190 



SOURCES: NORC calculations from Carttr Btglnnlngs tnrollmtnt fom and 1988 and 1989 survty 

data. 



NOTES: This tabit Includts strvlcts rtctlvtd fron Carttr Btglnnlngs and/or othtr sourcts. A 
participant could rtctlvt a naxlnun of 11 strvlcts (stt Tabit 3.6). 

Calculations for this tabit ustd stif-rtporttd data for all 621 txptrlntntals and all 
612 controls nho rtspondtd to both tht 1988 and 1989 survtys. Including thost with valuta of ztro for 
outcoMs and thost who wtrt asslgntd to Carttr Btglnnlngs but did not partlclpatt. 

Avtragt txptrlMtntal and control group outcomts rtporttd htrt by strvlct cattgory art 
adjusttd Mans fron stparatt llntar analysis of covarlanct proctdurts coi;tro111ng for 24 kinds of 
difftrtnct In characttrl sties btfort randon asslgnMnt (stt Ostit, 1975, p. 461; Cava, 1987). 
tecaust of rounding, thtrt may bt slight discrtpanclts In rtporttd i\m% and difftrtncts of thtst 
adjusttd Mans. 

*F ttsts wtrt applltd to difftrtncts tMng avtragt outcoMs by strvlct cattgory. Tht 
coluM labtltd "p" Is tht statistical signlflcanct Itvtl of tht difftrtncts tMng avtragt outcoMs: 
that Is, p Is tht probability that avtragt outcoMS art difftrtnt only btcaust of random trror. 
Statistical signlflcanct Itvtis art Indlcattd as *** • 1 ptrctnt or Itss; • 5 ptrctnt; 
* » 10 ptrctnt. 
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11 services attended a two- or four-year college by May 1989; 46.5 percent of the 90 
escperimentals who received 1 or 2 services went to college, more than double the rate for 
those ^o received none. As shown in the next two rows, additional services were associated 
with higher rates of college-going, although the increase in association with college-going from 
additional services was lower after the first 2 had been received. 

The right side of the table shows similar results within the control group. Getting no 
career services again was associated with a much lower rate of college-going, and again the 
marginal increase in association with college-going from additional services after the first 2 
was positive but much smaller. Of course, the caveats noted earlier must be borne in mind 
when interpreting these results: The number of services to be received was not assigned to 
sample members at random, and the unmeasured characteristics of those receiving different 
numbers of services were not the same. Because of possible selection bias, the direction of 
causation for interpreting this statistical association is therefore debatable. It cannot be said 
for sure whether using more services leads to more college-going or whether the college-bound 
tend to make sure to get more career services. However, the experimental evaluation results 
presented in Chapters 3 and 4 (especially the site impacts in Tables 3.11 and 4.4), showing that 
random assignment to greater availability of career services led to greater increases in college- 
going, makes it seem plausible that there is some causation from service receipt to college- 
going. 

The next section of Table C.7 shows the association between number of career services 
and length of stay in college. The no-service experimentals stayed an average of 2.2 months, 
and the no-service controls stayed an average of 23 months. This implies that those who did 
attend stayed nearly 9 months, since the more-than-three-fourths who did not attend each 
stayed zero months. In general, these length-of-stay figures, both for experimentals and for 
controls, mirror the results already presented on college attendance: Those who went to college 
at all tended to stay for a full 9 months. 

Toe final section of the table deals with attendance the end of survey follow-up, in 
May 1969. Because those who went to college at all tended to stay until the end of the school 
year, the estimates in this section are not much different from the estimates in the first section. 
Thus, 51.8 percent of experimentals who received 6 or more career services attended college 
during that month, compared to 22.4 percent of no-service experimentals. The rates of 
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attendance at the end of follow-up for those who received 1 or 2 services and for those who 
received 3, 4. or 5 services are between the extremes of 22.4 percent and 51.8 percent 
Although coUege-going rates generaUy are higher for groups who received more services, the 
rate for the group with 3, 4. or 5 services is a bit below that of the group with 1 or 2, though 
the difference is not statisticaUy significant ControU show a similar pattern, without the 
anomaly just noted. 

Table C8 presents results on the relationship between college-going outcomes and 
mentoring. As in Table C7, the 621 experimentals and 612 controU in the impact sample 
were analyzed separately. The first section of the table shows that, for experimentals, having 
a mentor is associated with a 55.5 percent rate of attendance at two- or four-year colleges, 
while not having a mentor is associated with a 47.1 percent rate of college-going. 

The right side of the table again shows similar results within the control group. Not 
having had a mentor is associated with a lower rate of college-going, but, within the control 
group, both those who did and did not have mentors show lower rates of college-going than 
similarly served experimentals. 

TTie next section of the table deals with length of stay and, as in the previous uble, the 
pattern of length of stay reflects the pattern of coUege attendance. The final section presents 
results on attendance at the end of foUow-up. Attrition seemed to be about 5 or 6 percentage 
points in each group, both among experimentals and among controls. 

Table C.9 addresses the possibility that mentoring enhances the effectiveness of other 
career services. Within categories of numbers of the 11 sendees received, those who had 
menton are distinguished from those who did not have mentors. Since mentoring was one 
of the 11 kinds of career service, it was impossible for any sample member to have had a 
mentor but to have received no career services. As in Table C7, the no-service group of 
experimentals went to college at a rate of about 23 percent Among experimentals who 
received one or two career services, 49.9 percent of those who had no mentor went to coUege, 
while 38J percent of those who had mentoring and one other service went to coUege. For 
those with more services in all, however, the story was very different Among experimentals 
who had mentoring as one of 3, 4, or 5 services, 522 percent went to coUege; among 
experimentals who had 3, 4, or 5 services, but not mentoring, 45.8 percent went to college. 
A simUar result obtained for those with 6 or more services. When mentoring was one of the 
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TABLE C.8 



EDUCATIONAL OUTCONES FOR EXPERINENTALS AND CONTROLS. 
BY MENTORING STATUS 



OutcoM and Ntntoring Status 


Erpariaentals 




Saapla 

Siza 




Controls 




Saapla 
Siza 


Evtr attandad a 2- or 4-y«er 
collaga, Juna IMS-Nay 1989 (%) 

Had no aantor 

Had a nantor 


47. l" 

55.5 


0.032 


222 
399 




45.6** 

53.6 


0.039 


336 
276 


Nunbar of mnths attandad 
a £• or 4-yaar collaga, 
Jirni IMS-Nay 1989 

Had no aantor 

Had a aantor 


3.9*** 
5.0 


0.004 


222 
399 




4.0 
4.4 


0.340 


336 
276 


Attandad a 2- or 4-yaar 
collaga during Nay 1989 (%) 

Had no aantor 

Had a aantor 


41. 6** 
50.6 


0.024 


222 
399 




40.9* 
47.6 


0.083 


336 
276 



SOURCES: NORC calculations from Carter Beginnings enrollment form and 1988 and 1989 survey 

data. 



NOTES: Calculations for this table used self-reported data for all 621 experlmtntals and all 
612 controls Mho responded to both the 1988 and 1989 surveys, Including those with values of zero for 
outcomes and those who were assigned to Career Beginnings but did not participate. 

Average experimental and control group oi les reported here by mentoring status are 
adjusted means from separate linear analysis of covarlance procedures controlling for 24 kinds of 
difference In characteristics before random assignment (ree Ostle, 1975, p. 461; Cave, 1987). 
Because of rounding, there may be slight discrepancies In reported sums and differences of these 
adjusted meant. 

*F tests were applied to differences among average outcomes by mentoring status. The 
column labeled "p" Is the statistical significance level of the differences among average outcomes: 
that Is, p It the probability that average outcomes are different only because of random error. 
Statistical tignlflcance levels are .ndlcated as » 1 percent or less; » 5 percent; 
* » 10 percent. 
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TABLE C.9 



EDUCATIONAL OUTCOMES FOR EXPERINENTALS AND CONTROLS. 
BY NUMBER OF SERVICES RECEIVED AND MENTORING STATUS 



OutcoMf Nunbtr of Strvlcts 
Rtcti^Mt tno Mtntorlng Status 


ExptrlMntals 




Saaplt 

Sizt 


Controls 




Samplt 

Sizt 


Evtr attofidad • 2- or 4-yMr 














co11«9t, 1M8-Itay 1989 (%) 














No strvlcts and no Mntor 


23.1 


0.306 


16 


27.7 


0.147 


27 


1 or 2 strvlcts and no mtntor 


49.9 




64 


45.9 




120 


1 or 2 strvlcts and a Mntor 


38.3 




26 


51.0 




2S 


3-5 strvlcts and no Mntor 


45.8 




88 


50.9 




138 


3-5 strvlcts and a Mntor 


52.2 




113 


49.8 




112 


6 or Mrt strvlcts and no Mntor 


53.0 




54 


39.3 




SI 


6 or Mrt strvlcts and a Mntor 


58.7 




260 


57.3 




139 


NuMtr of Mnths atttndtd 














a 2- or 4-ytar colltgt. 














Junt 1988-Nay 1989 














No strvlcts «.nd no Mntor 


2.2* 


0.062 


16 


2.3 


0.197 


27 


1 or 2 strvlcts and no Mntor 


4.4 




64 


3.9 




120 


1 or 2 strvlcts and a Mntor 


3.0 




26 


3.6 




2S 


3-5 strvlcts and no Mntor 


3.8 




88 


4.6 




138 


3-5 strvlcts and a Mntor 


4.5 




113 


4.1 




112 


6 or Mrt strvlce^ and no Mntor 


4.1 




54 


3.7 




SI 


6 or Mrt strvlcts and a Mntor 


5.3 




260 


4.8 




139 


Atttndtd a 2- or 4-ytar 














colltgt during Nay 1989 (%) 














No strvlcts and no Mntor 


22. 8* 


0.087 


16 


.7.7 


0.280 


27 


1 or 2 strvlcts and no Mntor 


50.1 




64 


40.6 




120 


1 or 2 strvlcts and a Mntor 


34.9 




26 


47.7 




2S 


3-S strvlcts and no Mntor 


39.1 




88 


45.2 




138 


3-5 strvlcts and a Mntor 


46.5 




113 


43.9 




112 


6 or Mrt strvlcts Mntor 


41.3 




54 


36.7 




SI 


6 or Mrt strvlcts <i a Mntor 


53.8 




260 


50.8 




139 



SOURCES: NDRC calculations from Carttr Btglnnlngs tnrollMnt form and 1988 and 1989 sunty 

data. 



NOTES: This tabit Includts strvlcts rtctlvtd from Carttr Btglnnlngs and/or othtr sourcts. A 
participant could rtctlvt a mxIwh of 11 strvlcts. Including Mntoring (stt TaLIt 3.6). 

Calculations 1 this tabit ustd stif-rtporttd data for all 621 txptrlMntals and all 
612 controls Mho rtspondtd to both tht 1988 and 1989 survtys. Including thost with valuta of ztro for 
outcoMs and thost Mho wtrt asslgntd to Carttr Btglnnlngs but did not parllclpatt. 

Avtragt txptrlMntal and control group outcoMS rtporttd htrt by strvlct cattgory and 
Mntoring status arc adjusttd Mans froM stparatt llntar analysis of covarlanct proctdurts 
controlling for 24 kinds of difftrtnct \^ characttrl sties btfort random asslgnMnt (stt Ostlt, 1975, 
p. 461; Cava, 1987). Btcaust of rounding, thtrt My bt slight discrtpanclts In rtporttd sums and 
difftrtncts of thtst adjusttd Mans. 

*F ttsts Mtrt applltd to tht inttractlon bttMttn Mntoring status and tht distribution 
of nwbtr of strvlcts rtctlvtd. Tht column labtltd "p" is tht statistical signlflcanct Itvtl of tht 
difftrtncts among avtragt outcoMs: that Is, p Is tht probability that avtragt outcoMs art difftrtnt 
only btcaust of random trror. Statistical signlflcanct Itvtls art 1nd4cattd as - 1 ptrctnt or 
lots; ■ 5 ptrct.* 2 * ■ 10 ptrctnt. 



services, experimentals went to college at a rate of 58.7 percent; when mentoring was not one 
of many services, 53.0 percent of experimentals went to college. 

The right side of the table shows similar but stronger results among highly served 
controls. While 39.3 percent of controls who had 6 or more services, but not mentoring, went 
to college, 57.3 percent of controls who had mentoring as one of their many services went to 
college. Among controls who had 3, 4, or 5 services, there was little difference in college- 
going between those who had mentors and those who did not. Among controls who had only 
1 or 2 services, when mentoring was one of them, they went to college at a rate of 51.0 
percent; when mentoring was not one of them, the rate was 45.9 percent. 

The next section of the table presents length-of-stay results. These generally agree with 
the results on college attendance and show similar patterns among experimentals and controls. 
The final section presents results for college attendance at the end of the follow-up, which, 
with one anomaly among experimentals, reflects the general pattern already discussed. 

Bearing in mind the caveats, these results may be suggestive of a ''synergistic" 
relationship between rrientoring and other services. Among experimentals and among controls, 
mentoring was associated with higher rates of college-going when it was one of 6 or more 
career services. A mentor may provide "case management** to help a disadvantaged young 
person persevere through the long process of getting information about colleges, taking 
entrance exams, writing applications, and arranging flnancial aid. It is plausible that having 
someone to play such a role would lead to increased college attendance, and Tables C.8 and 
C9 lend support to such views. 

VIL Summarv 

The Career Beginnings evaluation answers the question of whether the ready availability 
of more career services leads to more college-going among disadvantaged in-school youths. 
However, another question for programs targeting* disadvantaged college-bound youths is not 
whether Career Beginnings in particular raises their college attendance rates, but rather whether 
in general getting them career services leads to their going to college at higher rates. The 
reason the more general question cannot be answered directly by the Career Beginnings 
random assignment evaluation is that many members of the control group received similar 
career sen ces, mainly from sources other than Career Beginnings. 
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Concentrating especially on the association between service receipt and college-going, 
this appendix has taken a step back from the evaluation to look not at experimental-control 
differences but rather at the much wider variation in service receipt and college-going wUhin 
the control group and within the experimental group in order to explore this more general 
questioa Rates of college-going have been presented for those who received no career 
services and compared to rates for those who got some help. Tests of a few simple hypotheses 
about service receipt and outcomes were conducted, showing that increased service receipt is 
associated with increased college-going, that individual services are associated quite differently 
with rate of college-going, and that mentoring may servre as a catalyst that increases the 
association of other services with college-going. 

Because of the selection bias problem, these results are only suggestive. Since 
non-Career Beginnings services were not assigned to sample memben at random, measured 
and/or unmeasured characteristics of those who got different services were not the same. The 
results presented in this chapter were adjusted for differences in measured characteristics 
between sample members in different service categories. However, the analysis done here 
could not adjust for differences in unmeasured characteristics. Because groups who got 
different services may have differed, and likely did differ, in unmeasured characteristics, it 
cannot be determined from this analysis whether uving a service leads to college-going, or 
Mother the college-bound tend to make sure to iiv. ^T/ice, or whether someone else 
tends to make sure they get it More rigomus examinaiion of the associations discussed in this 
chapter is a subject for future research, in a context in which something closer to a low-service 
control group may be observed. 
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